Longtext Posts

Software requirements are rather straightforward - if we look at the requirements document, we see simple, declarative statements like "Users can log out," or "Users can browse and create topics." And that's when we're lucky enough to get an actual requirements document.

This is not legal advice

None of the following is intended to be legal advice. I am not a lawyer, have not even read all that many John Grisham novels, and am providing this as background for you to use. If you have actual questions, please take them to an actual lawyer. (Or you can try calling John Grisham, but I doubt he'd pick up.)

But there are other requirements in software engineering that aren't as cut-and-dried. Non-functional requirements related to things like maintainability, security, scalability and, most importantly for our purposes, legality.

For the sake of convenience, we're going to use "regulations" and other derivations of the word to mean "all those things that carry the weight of law," be they laws, rules, directives, court orders or what have you.

Hey, why should I care? Isn't this why we have lawyers?

Hopefully your organization has excellent legal representation. Also hopefully, those lawyers are not spending their days watching you code. That's not going to be fun for them or you. You should absolutely use lawyers as a resource when you have questions or aren't sure if something would be covered under a specific law. But you have to know when to ask those questions, and possess enough knowledge when your application could be running afoul of some rule or another.

It's also worthwhile to your career to know these things! Lots of developers don't, and your ability to point them out and know about them will make you seem more knowledgeable (because you are!). It will also make you seem more competent and capable than another developer who does not – again, because you are! This stuff is a skillset just like knowing Django.

While lawyers may be domain experts, they aren't always (especially at smaller organizations) and there are lots of regulations that specifically cover technology/internet-capable software that domain experts likely would not (and should not) be expected to be on top of. Further, if you are armed with foreknowledge, you don't have to wait for for legal review after the work has been completed.

Also, you know, users are people, too. Most regulations wind up being bottom-of-the-barrel expectations that user data is safeguarded and restricting organizations from tricking users into doing things they wouldn't have otherwise. In the same way I would hope my data and self-determination are respected, I also want to do the same for my users.

Regulatory environments

The difference in the regulatory culture between the US and the European Union is vast. I truly cannot stress how different they are, and that's an important thing to know about because it can be easy to become fluent in one and assume the other is largely the same. It's not. Trust me.

United States

The US tends, for the most part, to be a reactionary regulator. Something bad happens, laws or rules (eventually) get written to stop that thing from happening again.

Also, the interpretations of those rules tend to fluctuate more than in the EU, depending on things seemingly as random as which political party is in power (and controlling the executive branch, specifically) or what jurisdiction a lawsuit is filed in. We will not go in-depth into those topics, for they are thorny and leave scars, but it's important to note. The US also tends to give wide latitude to the defense of, "but it's our business model!" The government will not give a full pass on everything, but they tend to phrase things in terms of "making fixes" rather than "don't do that."

Because US regulations tend to be written in response to a specific incident or set of incidents, they tend for the most part to be very narrowly tailored or very broad ("e.g., TikTok is bad, let's give the government the ability to jail you for 20 years for using a VPN!"), leaving little guidance to those of us in the middle. This leaves lots of room for unintended consequences or simply failing to achieve the stated goals. In 2003, Congress passed the CAN-SPAM Act to "protect consumers and businesses from unwanted email." As anyone who ever looks at their spam box can attest, CAN-SPAM's acronym unfortunately seems to have meant "can" as in "grant permission," not "can" as in "get rid of."

European Union

In contrast, the EU tends to issue legislation prescriptively; that is, they identify a general area of concern, and then issue rules about both what you can and cannot do, typically founded in some fundamental right.

This technically is what the US does on a more circumspect level, but the difference is the right is the foundational aspect in the EU, meaning it's much more difficult to slip through a loophole.

From a very general perspective, this leads to EU regulations being more restrictive in what you can and can't do, and the EU is far more willing to punish punitively those companies who run afoul of the law.

Global regulations

There are few regulations that apply globally, and usually they come about backwards - in that a standard is created, and then adopted throughout the world.

Accessibility

In both the US and the EU, the general standard for digital accessibility is WCAG 2.1, level AA. If your website or app does not meet (most) of that standard, and you are sued, you will be found to be out of compliance.

In the US, the reason you need to be compliant comes from a variety of places. The federal government (and state governments) need to be compliant because of the Rehabilitation Act of 1974, section 508. Entities that receive federal money (including SNAP and NSF grants) need to be compliant because of the RA of 1974, section 504. All other publicly accessible organizations (companies, etc.) need to have their websites compliant because of the Americans with Disabilities Act and various updates. And all of the above has only arisen through dozens of court cases as they wound their way through the system, often reversing each other or finding different outcomes with essentially the same facts. And even then, penalties for violating the act are quite rare, with the typical cost being a) the cost of litigation, and b) the cost of remediation and compliance (neither of which are small, but they're also not punitive, either).

In the EU, they issued the Web Accessibility Directive that said access to digital information is a right that all persons, including those with disabilities, should have, so everything has to be accessible.

See the difference?

WCAG provides that content should be

• Perceivable - Your content should be able to be consumed in more than one of the senses. The most common example of this is audio descriptions on videos (because those who can't see the video still should be able to glean the relevant information from it).

• Operable - Your content should usable in more than one modality. This most often takes the form of keyboard navigability, as those with issues of fine motor control cannot always handle a mouse dextrously.

• Understandable - Your content should be comprehensible and predictable. I usually give a design example here, which is that the accessibility standard actually states that your links need to be perceivable, visually, as links. Also, the "visited" state is not just a relic of CSS, it's actually an accessibility issue for people with neurological processing differences who want to be able to tell at a glance what links they've already been to.

  Robust - Very broadly, this tenet states you should maximize your compliance with accessibility and other web standards, so that current and future technologies can take full advantage of them without requiring modification to existing content.

Anyway, for accessibility, there's a long list of standards you should be meeting. The (subjectively) more important ones most frequently not followed are:

1. Provide text alternatives for all non-text content: This means alt text for images, audio descriptions for video and explainer text for data/tables/etc. Please also pay attention to the quality – the purpose of the text is to provide a replacement for when the non-text content can't be viewed, so "picture of a hat" is probably not an actual alternative.

2. Keyboard control/navigation: Your site should be navigable with a keyboard, and all interactions (think slideshows, videos) should be controllable by a keyboard.

3. Color contrast: Header text should have a contrast ratio of 3:1 between the foreground and background; smaller text should have a ratio of 4.5:1.

4. Don't rely on color for differentiation: You cannot rely solely on color to differentiate between objects or types of objects. (Think section colors for a newspaper website: You can't just have all your sports links be red, it has to be indicated some other way.)

5. Resizability: Text should be able to be resized up to 200% larger without loss of content or functionality

6. Images of text: Don't use 'em.

7. Give the user control: You can autoplay videos or audio if you must, but you also have to give the user the ability to stop or pause it.

There are many more, but these are the low-hanging fruit that lots of applications still can't manage to pick off

PCI DSS

The Payment Card Industry Data Security Standard is a set of standards that govern how you should store credit card data, regulated by credit card companies themselves. Though some individual US states require adherence to the standards (and fine violators appropriately), federal and EU law does not require you to follow these standards (at least, not specifically these standards). However, the credit card companies themselves can step in and issue fines or, more critically, cut off access to their payment networks if they find the breaches egregious enough.

In most cases, organizations offload their payment processing to a third party (e.g., Stripe, Paypal), who is responsible for maintaining compliance with the specification. However, you as the merchant or vendor need to make sure you’re storing the data from those transactions in the manner provided by the payment processor; it’s not uncommon to find places that are storing too much data on their own infrastructure that technically falls under the scope of PCI DSS.

Some of the standards are pretty basic - don’t use default vendor passwords on hardware and software, encrypt your data transmissions. Some are more involved, like restricting physical access to cardholder data, or monitoring and logging access to network resources and data.

EU regulations

GDPR

The EU's General Data Privacy Regulation caused a big stir when it was first released, and for good reason. It completely changed the way that companies could process and store user data, and severely restricted what sort of shenanigans companies can get up to.

The GDPR states that individuals have the right to not have their information shared; that individuals should not have to hand over their information in order to access goods or services; and that individuals have further rights to their information even once it's been handed over to another organization.

For those of us on the side of building things, it means a few things are now requirements that used to be more "nice-to-haves."

• You must get explicit consent to collect data If you're collecting data on people, you have to explicitly ask for it. You have to specify exactly what information you're collecting, the reason you're collecting it, how long you plan on storing it and what you plan to do with it (this is the reason for the proliferation of all those cookie banners a few years ago). Furthermore, you must give your users the right to say no. You can't just pop up a full-screen non-dismissable modal that doesn't allow them to continue without accepting it.

• You can only collect data for legitimate purposes Just because someone's willing to give you data doesn't mean you're allowed to take it. One of my biggest headaches I got around GDPR was when a client wanted to gate some white papers behind an email signup. I patiently explained multiple times that you can't require an email address for a good or service unless the email address was required to provide said good or service. No matter how many times the client insisted that he had seen someone else doing the same thing, I stood firm and refused to build the illegal interaction.

• Users have the right to ask for the data you have stored, and to have it deleted Users can ask to see what data you have stored on them, and you're required to provide it (including, again, why you have that data stored). And, unless it's being used for legitimate processing purposes, you have to delete that data if the user requests it (the "right to be forgotten").

And all of this applies to any organization or company that provides a good or service to any person in the EU. Not just paid, either – it explicitly says that you do not have to charge money to be covered under the GDPR. So if your org has an app in the App Store that can be downloaded in Ireland, Italy, France or any other EU country, it and likely a lot more of your company's services will fall under GDPR.

As for enforcement, organizations can be fined up to €20 million, or up to 4% of the annual worldwide turnover of the preceding financial year, whichever is greater. Amazon Europe got docked €746 million for what was alleged "[manipulation of] customers for commercial means by choosing what advertising and information they receive[d]" based on the processing of personal data. Meta was fined a quarter of a billion dollars a few different times.

But it's not just the big companies. A translation firm got hit with fines of €20K for "excessive video surveillance of employees" (a fine that's practically unthinkable in the US absent cameras in a private area such as the bathroom), and a retailer in Belgium had to pay €10K for forcing users to submit an ID card to create a loyalty account (since that information was not necessary to creating a loyalty account).

Digital Markets Act

The next wave of regulation to hit the tech world was the Digital Markets Act. which is aimed specifically at large corporations that serve a “gatekeeping functionality” in digital markets in at least three EU countries. Although it is not broadly applicable, it will change the way that several major platforms will work with their data.

The directive’s goal is to break up the oversized share that some platforms have in digital sectors like search, e-commerce, travel, media streaming, and more. When a platform controls sufficient traffic in a sector, and facilitates sales between businesses and users, it must comply with new regulations about how data is provisioned and protected.

Specifically, those companies must:

• Allow third parties to interoperate with their services

• Allow businesses to access the data generated on the platform

• Provide advertising partners with the tools and data necessary to independently verify claims

• Allow business users to promote and conduct business outside of the platform

Additionally, the gatekeepers cannot:

• Promote internal services and products over third parties

• Prevent consumers from linking up with businesses off their platforms

• Prevent users from uninstalling preinstalled software

• Track end users for the purpose of targeted advertising without users’ consent

If it seems like these are aimed at the Apple App Store and Google Play Store, well, congrats, you cracked the code. The DMA aims to help businesses have a fairer environment in which to operate (and not be completely beholden to the gatekeepers), and allow for smaller companies to innovate without being hampered or outright squashed by established interests.

US regulations

The US regulatory environment is a patchwork of laws and regulations written in response to various incidents, and with little forethought for the regulatory environment as a whole. It’s what allows you as a developer to say, “Well, that depends …” in response to almost any question, to buy yourself time to research the details.

HIPAA

Likely the most well-known US privacy regulation, HIPAA covers almost none of the things that most people commonly think it does. We'll start with the name: Most think it's HIPPA, for Health Information Privacy Protection Act. It actually stands for Healthcare Insurance Portability and Accountability Act, because most of the law has nothing to do with privacy.

It is very much worth noting that HIPAA only applies to health plans, health care clearinghouses, and those health care providers that transmit health information electronically in connection with certain administrative or financial transactions where health plan claims are submitted electronically. It also applies to contractors and subcontractors of the above.

That means most of the time when people publicly refuse to comment on someone's health status because of HIPAA (like, in a sports context or something), it's nonsense. They're not required to disclose it, but it's almost certainly not HIPAA that's preventing them from doing so.

What is relevant to us as developers is the HIPAA Privacy Rule. The HIPAA privacy rule claims to "give patients more control over their health information, set boundaries on the use of their health records, establish appropriate safeguards for the privacy of their information."

What it does in practice is require that you have to sign a HIPAA disclosure form for absolutely every medical interaction you have (and note, unlike GDPR, that they do not have to let you say "no"). Organizations are required to keep detailed compliance policies around how your information is stored and accessed. While the latter is undoubtedly a good thing, it does not rise to the level of reverence indicated by its stated goals.

What you as a developer need to know about HIPAA is you need to have very specific policies (think SOC II [official link] [more useful link]) around data access, operate using the principle of least privileged access (only allow those who need to see PHI to be able to access it), and specific security policies related to the physical facility where the data is stored.

HIPAA’s bottom line is that you must keep safe Protected Health Information (PHI), which covers both basic forms of personally identifiable information (PII) such as name, email, address, etc., as well as any health conditions those people might have. This seems like a no-brainer, but it can get tricky when you get to things like disease- or medicine-specific marketing (if you’re sending an email to someone’s personal email address on a non-HIPAA-compliant server about a prostate cancer drug, are you disclosing their illness? Ask your lawyer!).

There are also pretty stringent requirements related to breach notifications (largely true of a lot of the compliance audits as well). These are not things you want to sweep under the rug. It’s true that HIPAA does not see many enforcement acts around the privacy aspects as some of the other, jazzier regulations. But health organizations also tend to err on the side of caution and use HIPAA-certified hosting and tech stacks, as any medical provider will be sure to complain about to you if you ask them how they enjoy their Electronic Medical Records system.

Section 230 of the Communications Decency Act

Also known as the legal underpinnings of the modern internet, Section 230 provides that "No provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider."

In practice, this means that platforms that publish user-generated content (UGC) will not be treated as the "publisher," in the legal sense, of that content for the purposes of liability for libel, etc. This does not mean they are immune from copyright or other criminal liabilities but does provide a large measure of leeway in offering UGC to the masses.

It's also important to note the title of the section, "Protection for private blocking and screening of offensive material." That's because Section 230 explicitly allows for moderation of private services without exposing the provider to any liability for failing to do so in some instances. Consider a social media site that bans Nazi content; if that site lets a few bad posts go through, it does not mean they are on the hook for those posts, at least legally speaking. Probably a good idea to fix the errors lest they be found guilty in the court of public opinion, though.

GLBA

The Graham-Leach-Biley Act is a sort of privacy protection policy for financial institutions. It doesn’t lay out anything particular novel or onerous - financial institutions need to provide a written privacy policy (what data is collected, how it’s used, how to opt-out), and provides some guidelines companies need to meet about safeguarding sensitive customer information. The most interesting, to me, requirement is Pretext Protection, which actually enshrines in law that companies need to have policies in place for how to prevent and mitigate social engineering attacks, both of the phishing variety as well as good old-fashioned impersonation.

COPPA

The Children's Online Privacy Protection Rule (COPPA, and yes, it’s infuriating that the acronym doesn’t match the name) is one of the few regulations with teeth, largely because it is hyperfocused on children, an area of lawmaking where overreaction is somewhat common.

COPPA provides for a number of (now) common-sense rules governing digital interactions that companies can have with children under 13 years old. Information can only be collected with:

• Explicit parental consent.

• Separate privacy policies must be drafted and posted for data about those under 13.

• A reasonable means for parents to review their children's data.

• Establish and maintain procedures for protecting that data, including around sharing that data.

• Limits on retention of that data.

• Prohibiting companies from asking for more data than is necessary to provide the service in question.

Sound weirdly familiar, like GDPR? Sure does. Wondering why only children in the US are afforded such protections? Us too!

FERPA

The Family Educational Rights Protection Act is sort of like HIPAA, but for education. Basically, it states that the parents of a child have a right to the information collected about their child by the school, and to have a say in the release of said information (within reason; they can't squash a subpoena or anything). When the child reaches 18, those rights transfer to the student. Most of FERPA comes down to the same policy generation around retention and access discussed in the section on HIPAA, though the disclosure bit is far more protective (again, because it's dealing with children).

FTC Act

The Federal Trade Commission Act of 1914 is actually the law that created the Federal Trade Commission, and the source of its power. You can think of the FTC as a quasi-consumer protection agency, because it can (and, depending on the political party in the presidency, will) go after companies for what aren't even really violations of law so much as they are deemed "unfair." The FTC Act empowers the commission to prevent unfair competition, as well as protect consumers from unfair/deceptive ads (though in practice, this has been watered down considerably by the courts).

Nevertheless, of late the FTC has been on a roll, specifically targeting digital practices. An excellent recent example was the settlement by Epic Games, makers of Fortnite. The FTC sued over a number of allegations, including violations of COPPA, but it also explicitly called out the company for using dark patterns to trick players into making purchases. The company’s practice of saving any credit cards used (and then making that card available to the kids playing), confusing purchasing prompts and misleading offers were specifically mentioned in the complaint.

CAN-SPAM

Quite possibly the most useless technology law on the books, CAN-SPAM (Controlling the Assault of Non-Solicited Pornography And Marketing Act) clearly put more time into the acronym than the legislation. The important takeaways are that emails need:

• Accurate subjects

• To disclose themselves as an ad

• Unsubscribe links

• A physical address for the company

And as your spam box will tell you, it solved the problem forever. This does not, however, mean you can ignore its strictures! As a consultant at a company that presumably wishes to stay on the right side of the law, you should still follow its instructions.

CCPA and Its Ilk

The California Consumer Privacy Act covers, as its name suggests, California residents in their dealings with technology companies. Loosely based on the GDPR, CCPA requires that businesses disclose what information they have about you and what they do with it. It covers items such as name, social security number, email address, records of products purchased, internet browsing history, geolocation data, fingerprints, and inferences from other personal information that could create a profile about your preferences and characteristics.

It is not as wide-reaching or thorough as GDPR, but it’s better than the (nonexistent) national privacy law.

The CCPA applies to companies with gross revenues totaling more than $25 million, businesses with information about more than 50K California residents, or businesses who derive at least 50% of their annual revenue from selling California residents’ data. There are similar measures that have already been made law in Connecticut, Virginia, Colorado, and Utah, as well as other states also considering relevant bills.

Other state regulations

The joy of the United States’ federalist system is that state laws can be different (and sometimes more stringent!) than federal law, as we see with CCPA. It would behoove you to do a little digging into the state regulations when you’re working with specific areas — e.g., background checks, where the laws differ from state to state, as even though you’re not based there, you may be subject to its jurisdiction.

There are two different approaches companies can take to dealing with state regulations: Either treat everyone under the strictest regulatory approach (e.g., treat every user like they’re from California) or make specific carve-outs based on the state of residence claimed by the user.

It is not uncommon, for example, to have three or four different disclosures or agreements for background checks ready to show a user based on what state they reside in. The specific approach you choose will vary greatly depending on the type of business, the information being collected, and the relevant state laws.

How to implement

Data compliance is critical, and the punitive aspects of GDPR’s enforcement means your team must have a solid strategy for compliance.

The most important aspect of dealing with any regulatory issue is first knowing what’s required for your business. Yes, you’re collecting emails, but to what end? If that data is necessary for your business to function, then you have your base-level requirements.

Matching those up against the relevant regulations will provide you with a starting point from which you can begin to develop the processes, procedures and applications that will allow your business to thrive. Don’t rely on “that’s how we’ve always done it” or “we’ve seen other people do x” as a business strategy.

The regulatory environment is constantly shifting, and it’s important to both keep abreast of changes as well as always knowing what data and services are integral to your business’s success. Keeping up with the prevalent standards will aid you not only in not getting sued, but also ensuring your companies that you’re a trustworthy and reliable partner.

How to keep up

It all seems a little daunting, no?

But you eat the proverbial regulatory elephant the same way you do any other large food item: one bite at a time. In the same way you didn’t become an overnight expert in securing your web applications against cross-site scripting attacks or properly manage your memory overhead, becoming a developer who’s well-versed in regulatory environments is a gradual process.

Now that you know about some of the rules that may apply to you, you know what to keep an eye out for. You know potential areas to research when new projects are pitched or started, and you know where to ask questions. You know to both talk to and listen to your company’s legal team when they start droning on about legalistic terms

People always seem confused by the title of this: "What does scrum have to do with measuring productivity?" they ask. And I smile contentedly, because that's the whole point.

Scrum is supposed to be a system for managing product work, iterating and delivering value to the customer. What usually winds up happening is scrum gets used for the management of software development work as a whole, from decisions about promotion to hiring and firing to everything else. That's not what scrum is designed to do, and it shows.

Now, I love to talk about process improvement, completely agnostic of whatever process framework you're using. I would much rather have a discussion about the work you're doing and what blockers you're hitting rather than discussing abstract concepts.

However, if you keep running into the same issues and blockers over and over again, it's usually worth examining your workflows to find out if you're actually applying the theory behind your framework to the actual work you're doing. The concept of Agile specifically is not about the processes involved, but you need to know and understand the rules before you should feel comfortable breaking them.

Processes

I want to start with a quick overview of a few key terms to make sure everyone's on the same page.

• Waterfall


  In waterfall development, every item of work is scheduled out in advance. This is fantastic for management, because they can look at the schedule to see exactly what should be worked on, and have a concrete date by which everything will done.

  This is horrible for everyone, including management, because the schedule is predicated upon developers being unerring prophets who are able to forecast not only the exact work that needs to be done to develop a release, but also the exact amount of time said work will take.

  The ultimate delimiter of the work to be done is the schedule - usually there’s a specific release date (hopefully but not always far out enough to even theoretically get all the work done); whatever can get done by that date tends to be what’s released.

  Waterfall also suffers greatly because it’s completely inflexible. Requirements are gathered months ahead of time; any changes require completely reworking the schedule, so changes are frowned upon. Thus, when the product is released, it’s usually missing features that would have been extremely beneficial to have.

• Agile

  Agile can be viewed as a direct response to waterfall-style development; rather than a rigid schedule, the agile approach embraces iteration and quick releases. The three primary “laws” of agile are:

  • Law of the customer - The customer is the number one priority. Rather than focusing on hitting arbitrary milestones or internal benchmarks, agile teams should be focused on delivering products to customers that bring them additional value. A single line of code changed can be more worthwhile than an entirely new product if that line brings extra value to the customer.

  • Law of small teams - Developers are grouped into small teams that are given autonomy in how they implement the features they’re working on. When work is assigned to a team, it’s not done so prescriptively. In the best agile teams, the assignment is, “Here’s the problem we have, go solve it.”

  • Law of the network - There are differing interpretations on how to implement this, but essentially I view of the network as “the whole organization has to buy in to what the agile teams are doing.” The entire organization doesn’t need to have the same structure as the agile teams, but neither can it be structured in a manner antithetical to the processes or outcomes.
    The easiest counterexample is the entire dev department is using scrum, but the CTO still feels (by virtue of their title) the ability to step in and make changes or contribute code or modify stories on a whim. Just because the CTO is the manager doesn’t mean they have full control over every decision. Basically, law of the network means “respecting the agile method, even if you’re not directly involved.”

  It’s worth noting that agile is a philosophy, not a framework in and of itself. Both kanban and scrum are implementations of the agile philosophy.

• Kanban

  This is usually the most confusing, because both scrum and kanban can use kanban boards (the table of stories, usually denoted by physical or virtual “post-its” that represent the team’s work). Kanban board splits up work into different “stages” (e.g., to-do, doing, done), and provides a visual way to track progression of stories.

  The primary difference between scrum and kanban as a product development methodology is that kanban does not have specific “sprints” of work - the delimiter of work is how many items are in a given status at a given time. For example, if team limits “doing” to four cards and there are already four cards in there, no more can be added until one is moved along to the next stage (usually this means developers will pair or mob on a story to get it through).

• Scrum

  Scrum, by contrast, delimits its work by sprints. Sprints are the collection of work the team feels is necessary to complete to deliver value. They can be variable in their length (though in practice, they tend to be a specified time length, which causes its own issues).

  Scrum requires each team to have at least two people - a product owner and a scrum master. Usually there are also developers, QA and devops people on the team as well, but at a minimum you need the PO and SM.

  The product owner has the vision for what the product should be - they should be in constant contact with customers, potential customers and former customers to figure out how value can be added. The scrum master’s job is to be the sandpaper for the developers - not (as the name implies) their manager or boss, but the facilitator for ceremonies and provide coaching/guidance on stories and blockers.

Other reasons processes fail

I will note that a lot of the reasons I will list below may also apply to other product management methodologies; however, I’m specifically limiting the scope to how they impact scrum teams.

Lack of product vision

I don’t want to lay the blame entirely on product owners for this issue - very often the problem is with how the role is designed and hired for. Product owners should be the final arbiters for product decisions. They should absolutely consult design, UX and customer service experts for their opinions, but the decisions ultimately lies with them.

Unfortunately, the breadth of skills required to be a good product owner are not in abundant supply, and product owners are, bafflingly, often considered afterthoughts at many organizations.

More than specific skills, though, product owners need to have a vision for what the product could be, as well as the flexibility to adapt that vision when new information comes in. Usually, this requires domain knowledge (that can be acquired, but needs to be done so systematically and quickly upon hiring), steadfastness of conviction and the ability to analyze data properly to understand what customers want.

Far too often product owners essentially turn into feature prioitizers, regurgitating nearly everything customers say they want and assigning a ranking to it. This often comes at the expense of both the product’s conceptual integrity as well as relationships with developers, who are supposed to be given problems to solve, not features to develop. This is the classic feature factory trap.

Mistake the rules for the reason

Far too often, people will adopt the ceremonies or trappings of scrum without actually accepting an agile mindset. This is where my favorite tagline, “that’s just waterfall with sprints” comes from.

If you’ve ever started a project by first projecting and planning how long it’s going to take you to deliver a given set of features, congratulations, you’re using waterfall.

To use scrum, you need to adopt the iterative mindset to how you view your product. If you’re developing Facebook, you don’t say, “we’re going to build a system that allows you to have an activity feed that shows posts from your friends, groups and advertisters, and have an instant messaging product, and ..”

Instead, you’d say, “we’re going to develop a platform that helps people connect to one another.” Then you’d figure out the greatest value you can add in one sprint (e.g., users can create profiles and upload their picture.). You know once you have profiles you’ll probably want the ability to post on others’ profiles, so that’s in the backlog.

That’s it. That’s the planning you do. Because once those releases get into customers’ hands, you’ll then have better ideas for how to deliver the next increment of value.

Simply because an organization has “sprints” and a “backlog” and do “retros” doesn’t mean its’ using scrum, it means it’s using the language of scrum.

Lack of discipline/iteration

Tacking on to the last point, not setting up your team for success in an agile environment can doom the product overall. Companies tend to like hiring more junior developers, because they’re cheaper, but not realizing that a junior developer is not just a senior developer working at 80% speed. Junior developers need to have mentoring and code reviews, and those things take time. If the schedule is not set up to allow for that necessary training and code quality checks to happen, the product will suffer overall.

Similarly, development teams are often kept at a starting remove from everyday users and their opinions/feedback. While I by no means advocate a direct open firehose of feedback, some organizations don’t ever let their devs see actual users using the product, which creates a horrible lack of feedback loop from a UX and product design perspective.

Properly investing in the team and the processes is essential to any organization, but especially one that uses scrum.

Lack of organizational shift

The last ancillary reason I want to talk about in terms of scrum failure is aligning the organization with the teams that are using scrum (we’re back to the law of network, here). Scrum does not just rely on the dev team buying in, it also requires the larger organization to at least respect the principles of scrum for the team.

The most common example of this I see is when the entire dev department is using scrum, but the CTO still feels (by virtue of their title) the ability to step in and make changes or contribute code or modify stories on a whim. Just because the CTO is the manager doesn’t mean they have full control over every decision. Removing the autonomy for the team messes with the fundamental principles of scrum, and usually indicates there will be other issues as well (and I guarantee that CTO will also be mad when the scrum is now unbalanced or work doesn’t get done, even though they’re the direct cause).

No. 1 reason scrum fails: It’s used for other purposes

By far, the biggest reason I see scrum failing to deliver is when the ceremonies or ideas or data generated by scrum gets used for something other than delivering value to the end users.

It’s completely understandable! Management broadly wants predictability, the ability to schedule a release months out so that marketing and sales can create content and be ready to go.

But that’s not how scrum works. Organizations are used to being able to dictate schedules for large releases of software all at once (via waterfall), and making dev deliver on those schedules. If you’re scheduling a featureset six months out, it’s almost guaranteed you’re not delivering in an agile manner.

Instead of marketing-driven development, why not flip the script and have development-driven marketing? There is absolutely no law of marketing that says you have to push a new feature the second it’s generally available. If the marketing team keeps up with what’s being planned a sprint in an advance, that means they’d typically have at least a full month of leadtime to prepare materials for release.

Rather than being schedulable, what dev teams should shoot for is reliability and dependability. If the dev team commits to solving an issue in a given sprint, it’d better be done within that sprint (within reason). If it’s not, it’s on the dev team to improve its process so the situation doesn’t happen again.

But why does scrum get pulled off track? Most often, it’s because data points in scrum get used to mean something else.

Estimates

The two hardest problems in computer science are estimates, naming things, and zero-based indexes. Estimates are notoriously difficult to get right, especially when developing new features. Estimates get inordinately more complex when we talk about story pointing.

Story points are a value assigned to a given story. They are supposed to be relative to other stories in the sprint - e.g., a 2 is bigger than a 1, or a medium is bigger than a small, whatever. Regardless of the scale you’re using, it is supposed to be a measure of complexity for the story for prioritization purposes only.

Unfortunately, what usually winds up happening is teams adopt some sort of translation scale (either direct or indirect), something like 1 = finish in an afternoon, 2 = finish in a day, 3 = multiple days, 5 = a week, etc. But then management wants to make sure everyone is pulling their fair share, so people are gently told that 10 is the expectation for the number of points they should complete in a two-week sprint, and now we are completely off the rails.

Story points are not time estimates. Full stop.

It’s not a contract, you’re not a traffic cop trying to make your quota. Story points are estimates of the complexity of a story for you to use in prioritization. That’s it.

I actually dislike measuring sprint velocity sprint-to-sprint, because I don’t think it’s helpful in most cases. It actually distorts the meaning of a sprint. Remember, sprints are supposed to be variable in length; if your increment of value is small, have a small sprint. But because sprint review and retro have to happen every second Friday, sprints have to be two weeks. Because the sprint is two weeks, now we have two separate focii, and the scrum methodology drifts further and further away.

Campbell’s law is one of my favorite axioms. Paraphrased, it states:

The more emphasis placed on a metric, the more those being measured will be incentivized to game it.

In the case above, if developers are told they should be getting 10 points per sprint, suddenly their focus is no longer on the customer. It’s now on the number of story points they have completed. They may be disincentivized to pick up larger stories, fearing they might get bogged down. They’re almost certainly going to overestimate the complexity of stories, because now underestimates mean they’re going to be penalized in terms of hitting their targets.

This is where what I call the Concilio Corollary (itself a play on the uncertainty principle) comes into play:

You change the outcome of development by measuring it.

It’s ultimately a question of incentives and focus. If you start needing to worry about metrics other than “delivering value to the user,” then your focus drifts from same. This especially comes into play when organizations worry about individual velocity.

I don’t believe in the practice of “putting stories down” or “pick up another story when slightly blocked.” If a developer is blocked, it’s on the scrum master and the rest of the team to help them get unblocked. But I absolutely understand the desire to do so if everybody’s expected to maintain a certain momentum, and other people letting their tasks lie to help you is detrimental to their productivity stats. How could we expect teamwork to flourish in such an environment?

So how do we measure productivity?

Short answer: don’t.

Long answer: Don’t measure “productivity” as if it’s a value that can be computed from a single number. Productivity on its own is useless.

I used to work at a college of medicine, and after a big website refresh they were all excited reporting how many pageviews the new site was getting. And it makes sense, because when we think of web analytics, we think page views and monthly visitors and time on site, all that good stuff.

Except … what’s the value of pageviews to a college? They’re not selling ads, where more views works out to more money. In fact, the entire point of the website was to get prospective students to apply. So rather than track “how many people looked at this site,” what they should have been doing was looking at “how many come to this site and then hit the ‘apply now’ button,” and comparing that to the previous incarnation.

First, you need to figure out what the metrics are being used for. There are any number of different reasons you might want to measure “productivity” on a development team. Some potential reasons include performance reviews, deciding who to lay off, justifying costs, figuring out where/whether to invest more, or fixing issues on the development team.

But each of those reasons has a completely different dataset you should be using to make that decision. If you’re talking about performance reviews, knowing the individual velocity of a developer is useless. If it’s a junior, taking on a 5-point story might be a huge accomplishment. If you’re looking at a principal or a senior, you might actually expected a lower velocity, because they’re spending more time pairing with other developers to mentor them or help them get unblocked.

Second, find the data that answers the question. When I worked at a newspaper, we used to have screens all over the place that showed how many pageviews specific articles were getting. Except, we didn’t sell ads based on total pageviews. We got paid a LOT of money to sell ads to people in our geographical area, and a pittance for everything else. A million pageviews usually meant we had gone viral, but most of those hits were essentially worthless to us. To properly track and incentivize for best return, we should have been tracking local pageviews as our primary metric.

Similarly, if you’re trying to justify costs for your development team, just throwing the sprint velocity out there as the number to look at might work at the beginning, but that now becomes the standard you’re measured against. And once you start having to maintain features or fix bugs, those numbers are going to go down (it’s almost always easier to complete a high-point new-feature story than a high-point maintenance story, simply because you don’t have to understand or worry about as much context).

There are a number of newer metrics that have been proposed as standards that dev teams should be using. I don’t have an inherent problem with most of these metrics, but I do want to caution not to just adopt them wholesale as a replacement for sprint velocity. Instead, carefully consider what you’re trying to use the data for, then select those metrics that provide that data. Those metrics are SPACE and DORA. Please note that these are not all individual metrics; some of them (such as “number of handoffs”) are team-based.

SPACE

• Satisfaction and well-being

◦ This involves developer satisfaction surveys, analyzing retention numbers, things of that nature. Try to quantify how your developers feel about their processes.

• Performance

◦ This might include some form of story points shipped, but would also include things like number and quality of code reviews.

• Activity

◦ Story points completed, frequency of deployments, code reviews completed, or amount of time spent coding vs. architecting, etc.

• Communication/collaboration

◦ Time spent pairing, writing documentation, slack responses, on-call/office hours

• Efficiency/flow

◦ Time to get code reviewed, number of handoffs, time between acceptance and deployment

DORA

DORA, or DevOps Research and Assessment, are mostly team-based metrics. They include:

• Frequency of deployments

• Time between acceptance and deployment

• How frequently deployments fail

• How long it takes to recover/restore from failed

Focus on impact

But all of these metrics should be secondary, as the primary purpose of a scrum team is to deliver value. Thus, the primary metrics should measure direct impact of work: How much value did we deliver to customers?

This can be difficult to ascertain! It requires a lot of setup and analysis around observability, but these are things that a properly focused scrum team should already be doing. When the dev team is handed a story for a new feature, one factor of that story should be success criterion: e.g., at least 10% of active users use this feature in the first 10 days. That measurement should be what matters most. And failing to meet that mark doesn’t mean the individual developer failed, it means some underlying assumption (whether it’s discoverability or user need) is flawed, and should be corrected for the next set of iterations.

It comes down to outcome-driven-development vs. feature-driven-development. In scrum, you should have autonomous teams working to build solutions that provide value to the customer. That also includes accountability for the decisions that were made, and a quick feedback loop coupled with iteration to ensure that quality is being delivered continuously.

TL;DR

In summation, these are the important bits:

• Buy in up and down the corporate stack - structure needs to at least enable the scrum team, not work against it

• Don’t estimate more than you need to, and relatively at that

• Know what you’re measuring and why

Now, I know individual developers are probably not in a position to take action at the level “stop using metrics for the wrong reasons.” That’s why I have a set of individual takeaways you can use.

• Great mindset for performance review

◦ I am a terrible self-promoter, but keeping in mind the value I was creating made it easy for me come promotion time to say, “this is definitively what I did and how I added value to the team.” It made it much easier for me than trying to remember what specific stories I had worked on or which specific ideas were mine.

• Push toward alignment

◦ Try to push your leaders into finding metrics that answer the questions they’re actually asking. You may not be able to get them to abandon sprint velocity right off the bat, but the more people see useful, actionable metrics the less they focus on useless ones.

• Try to champion customer value

◦ It’s what scrum is for, so using customer value as your North Star usually helps cut through confusion and disagreement.

• Get better at knowing what you know / don't know

◦ This is literally the point of sprint retros, but sharing understanding of how the system works will help your whole team to improve the process and produce better software.

Since this post is long enough on its own, I also have a separate post from when I gave this talk in Michigan of questions people asked and my answers.

I grew up on Clean Code, both the book and the concept. I strove for my code to be “clean,” and it was the standard against which I measured myself.

And I don’t think I was alone! Many of the programmers I’ve gotten to know over the years took a similar trajectory, venerating CC along with Code Complete and Pragmatic Programmer as the books everyone should read.

But along the way, “clean” started to take on a new meaning. It’s not just from the context of code, either; whether in interior design or architecture or print design, “clean” started to arise as a synonym for “minimalism.”

This was brought home to me when I was working with a junior developer a couple years ago. I refactored a component related to one we working on together to enable necessary functionality, and I was showing him the changes. This was a 200-line component, and he skimmed it about 45 seconds before saying “Nice, much cleaner.”

And it bugged me, but I wasn’t sure why. He was correct - it was cleaner, but it felt like that shouldn’t have been something he was accurately able to identify simply by glancing at it. Or at least, if that was the metric he was using, “clean” wasn’t cutting it.

Because the fact of the matter is you can’t judge the quality of code without reading it and understanding what it’s trying to do, especially without considering it in the context of its larger codebase. You can find signifiers (e.g., fewer lines of code, fewer methods in a class), but “terse” is not a direct synonym of “clean.” Sometimes less code is harder to understand or maintain than more code.

I wanted to find an approach, a rubric, that allowed for more specificity. When I get feedback, I much prefer hearing the specific aspects that are being praised or need work on - someone telling me “that code’s clean” or not isn’t particularly actionable.

So now I say code should be Comprehensible, Predictable and Maintainable. I liked those three elements because they’re important on their own, but also each builds on the others. You cannot have predictable and maintainable code unless it’s also comprehensible, for example.

• Comprehensible - People other than the author, at the time the code is written, can understand both what the code is doing and why.

• Predictable - If we look at one part of the code (a method, a class, a module), we should be able to infer a number of properties about the rest.

• Maintainable - Easy to modify and keep up, as code runs forever

Comprehensibility is important because we don’t all share the context - even if you’re the only person who’s ever going to read the code, the you of three weeks from now will have an entirely different set of issues you’re focusing on, and will not bring the same thoughts to bear when reasoning about the code. And, especially in a professional context, rare is the code that’s only ever read by one other person.

Predictability speaks to cohesion and replicability across your codebase. If I have a method load on a model responsible for pulling that object’s information from the database, all the other models should use load when pulling object info from the DB. Even though you could use get or loadFromDb or any number of terms that are still technically comprehensible, the predictability of using the same word to mean the same thing reduces overall cognitive load when reasoning about the application. If I have to keep track of which word means the action I’m trying to take based on which specific model I’m using, that’s a layer of mental overhead that’s doing nothing toward actually increasing the value or functionality of the software.

Maintainability is the sort of an extension of comprehensibility - how easy is it the code to change or fix down the road? Maintainability includes things like the “open to extension, closed to modification” principle from SOLID, but also things like comments (which we’ll get to, specifically, later on). Comprehensibility is focused on the “what” the code is doing, which often requires in-code context and clear naming. Maintainability on the other hand, focuses on the “why” - so that, if I need to modify it later on, I know what the intent of the method/class/variable was, and can adjust accordingly.

The single most important aspect of CPM code is naming things. Naming stuff right is hard. How we name things influences how we reason about them, how we classify them, and how others will perceive them. Because those names eventually evolve to carry meaning on their own, which can be influenced by outside contexts, and that whole messy ball of definition is what the next person is going to be using when they think about the thing.

I do believe most programmers intellectually know the importance of naming things, but it’s never given the proper level of respect and care its importance would suggest. Very rarely do I see code reviews that suggest renaming variables or methods to enhance clarity - basically, the rule is if it’s good enough that the reviewer understands it at that moment, that’s fine. I don’t think it is.

A class called User should contain all the methods related to the User model. This seems like an uncontroversial stance. But you have to consider that model in the context of its overall codebase. If there is (and there should be) also a class called Authorization in that codebase,  there are already inferences we should be able to draw simply from the names of those two things.

We should assume User and Authorization are closely related; I would assume that some method in Authorization is going to be responsible for verifying that the user of the application is a User allowed to access parts of the application. I would assume these classes are fairly tightly coupled in some respects, and it would be difficult to use one without the other, in some respect.

Names provide signposts and architecture hints of the broader application, and the more attuned to them you are (as both a writer and reader of code), the more information will be able to be conveyed simply by paying attention to them.

If naming is the single most important aspect of CPM, the single most important aspect of naming things is consistency. I personally don’t care about most styling arguments (camelCase vs. snake_case, tabs vs. spaces, whatever). If there’s a style guide for your language or framework, my opinion is you should follow it as closely as possible, deviating only if there’s an actual significant benefit to doing so.

Following style conventions has the two advantages: allowing for easier interoperability of code from different sources, and enabling the use of linters and formatters.

Code is easier to share (both out to others and in from others) if they use same naming conventions and styles, because you’re not adding an extra layer of reasoning atop the code. If you have to remember that Library A uses camelCase for methods but Framework B uses snake_case, that’s however large a section of your brain that is focusing on something other than the logic of what the code is doing.

And enabling linters and formatters means there’s a whole section of code maintenance you no longer have to worry about - you can offload that work to the machine. Remember, computers exist to help us solve problems and offload processing. A deterministic set of rules that can be applied consistently is literally the class of problems computers are designed to handle.

Very broadly, my approach to subjective questions is: Be consistent. Anything that doesn’t directly impact comprehensibility is a subjective decision. Make a decision, set your linter or formatter, and never give another thought to it. Again, consistency is the most important aspect of naming.

But a critically under-appreciated aspect of naming is the context of the author. Everyone sort of assumes we all share the same context, in lots of ways. “Because we work on the same team/at the same company, the next developer will know the meaning of the class PayGrimples.” That may be very broadly true, in that they’ve probably heard of PayGrimples, but it doesn’t mean they share the same context.

A pop-culture example of this is pretty easy - think of the greatest spaceship pilot in the universe, one James Tiberius Kirk. Think about all his exploits, all the strange new worlds he’s discovered. Get a good picture of him in your head.

Which one did you pick? Was it The Original Series’ William Shatner? The new movies’ Chris Pine? Or was it Strange New Worlds’ Paul Wesley?

You weren’t wrong in whatever you picked. Any of those is a valid and correct answer. But if we were talking about Kirk in conversation, you likely would have asked to clarify which one I meant. If we hadn’t, we could talk about two entirely different versions of the same concept indefinitely until we hit upon a divergence point when one of us realized.

Code has that same issue, except whoever’s reading it can’t ask for that clarification. And they can only find out they’re thinking about a different version of the concept if they a) read and digest the code in its entirety before working on it, or b) introduce or uncover a bug in the course of changing it. So when we name things, we should strive for the utmost clarity.

⛔ Unclear without context

type User = {
	id: number;
	username: string;
	firstName: string;
	lastName: string;
	isActive: boolean;
}

The above is a very basic user model, most of whose properties are clear enough. Id, username, firstName and lastName are all pretty self-explanatory. But then we get to the boolean isActive.

This could mean any number of things in context. They include, but are not limited to:

• The user is moving their mouse on the screen right now

• The user has a logged-in session

• The user has an active subscription

• The user has logged in within the last 24 hours

• The user has performed an authenticated activity in the last 24 hours

• The user has logged in within the last 60 days

All of those are things we may want to know about the user of any application, depending on what we’re trying to do. Even similar-sounding events with the same time horizon (logged in within the last 24 hours vs. authenticated activity in the last 24 hours) give us different information - I can infer the maximum age of the authentication token in the logged-in case, but without knowing the token exchange process, I cannot make the same inference for authenticated activity.

So why not just provide the meaning with the name?

✅ Clarity without context

type User = {
	id: number;
	username: string;
	firstName: string;
	lastName: string;
	loggedInPrevious24Hours: boolean;
}

Clarity comes through naming things explicitly. Ambiguity is the enemy of clarity, even when you assume the person reading the code should know something.

It’s reasonable to assume that the people reading your code are developers - that is, people familiar with coding concepts. Every other context (industry/domain, organization) is not a safe assumption. Therefore, if you have names or terms that are also used in coding, you should clarify the other meaning. (You should do this generally, as well, but specifically with programming-related terms.)

⛔ Ambiguity kills comprehension

class Class {}
class Post {} 

The word “class” is generally understanding in programming as an object-oriented prototype. Outside of programming, it could refer to a classroom of children; a classification system; a group of children in the same grade (e.g., junior class); or a social hierarchy (e.g., upper-class, lower-class).

Post is even worse, because it can be a verb or a noun even in a programming context. Blogs usually have posts, but you can also post content (or the HTTP verb, POST). Non-tech-wise, we have places to which you can be sent (“I’m being sent to our post in London”), referring to the mail system, or even structural support for fences.

✅ Specificity aids everyone

class Classroom {}
class BlogPost {}

All of this is important because being clear matters more than being concise or clever. After consistency, the most important aspect of naming is being descriptive. The name should describe what the code is doing (vs. why or how) - what a method is doing, or what purpose a variable serves.

For the most part, Classes should be nouns, because they’re describing their domain of influence. Methods should include verbs, because they’re performing actions. Variables should be nouns, reflective of whatever purpose they’re serving.

If you find yourself struggling with the length of your names of methods, variables or classes, that’s not a bad thing. It’s usually a sign you need to consider refactoring (more on this a bit later).

To the point of clarity, be sure to use properly spelled real words and names.

⛔ Abbreviations and shortcuts

class DateUtil {
  static function dateStrFrmo(date: Date): string { ... }
}

Humans have surprisingly good short- and medium-term recall around words and names. Using real words and names makes the concept easier for us to reason about, and easier to keep track of in our heads.

I took the example above from a GitHub code search. I think the original example may have been written by a native German speaker, because if we assume “Frmo” is supposed to be “From,” it’s using the German sentence structure that puts the verb at the end of the sentence. That makes sense! But if someone isn’t familiar with that sentence construction, the name of the method becomes functionally useless.

The misspelling part is important in two respects: one, it can introduce confusion (is it supposed to be “from” or ”form”?). The other is relying on the computer - searches, within the IDE or if you’re GREPing, are looking for specific terms. If it’s spelled wrong, it’s not going to get caught in the search.

✅ Use properly spelled real words and names

class DateUtil {
  static function getStringFromDate(date: Date): string { ... }
}

Here we’ve modified it so we essentially have an English sentence - get the string from the date. I know what’s being passed in (the date), and I know what’s coming out (the string), and I know overall what’s happening (I’m getting the string from the date).

Beyond naming, there is one other “big” rule that gets us to comprehensible, predictable and maintainable code, an old adage: “Keep it simple, sweetheart.” I’m not speaking to system complexity here - your overall architecture should be as complex as needed to do the job. It’s closer to SOLID’s single-responsibility principle, writ large: Every module, every class, every method, every variable should have one job.

To our earlier example of Users and Authorization, users will take care of the users while authorization handles auth. Neither of them should care about the internal workings of the other; Authorization just needs to know it can call User::load to return the user object.

At the method level, this is how we keep our names to a manageable length. You should be able to describe what the method in a very short sentence. If you need more length (or you leave out things it’s doing), it’s probably a sign that the method is trying to do too much.

Smaller methods enable reusability - if the method is only doing a specific thing, we are more likely to be able to use it somewhere else. If the method is doing multiple things, we’d likely need to add a parameter in the other cases where we want to use it, because we don’t want all of those things to happen all the time.

Keeping each method to a single task means we can decompose complex methods into multiple individual methods. This also makes it easier to read the code.

Literally just reading the names of methods allows us to infer what’s going on, divorced of context. For the example below, we would know from the file name this is Typescript, and I’ll give one hint that it’s frontend.

✅ Keep it simple, even for complex actions

function constructor() {
    this.assignElements();
    this.setInterval();
    this.getNewArt();
    this.listenForInstructions();
}

Initializing this class assigns elements and sets an interval (meaning there are actions that happen on a set schedule); then we get new art, and listen for instructions. Without even knowing the name of the class, we can pretty confidently assume this has to do with art, and that art gets changed frequently (hence the interval). But there also appears to be a manual interruption possible, with listen for instructions.

If we were debugging an issue related to keyboard commands, I would first look to listenForInstructions. If there’s an issue with art not showing up, I would check getNewArt.

Each method is narrowly scoped, even if a lot happens. Keeping things simple aids comprehension and predictability, but it’s also vital for maintainability. It makes it much easier to write tests.

We cannot confidently change code without tests. If we’re making modifications to code without tests, we can read, guess and hope that it won’t create any downstream issues, but unless we know exactly what should happen with a given method in all the ways it can be used, we cannot be certain of the impact of any change. A downstream issue is the definition of a regression; determining the output of a method in changing circumstances is the definition of a test. Thus why we test to avoid regressions.

A good unit test is like a science experiment - a hypothesis is proffered, and borne out or disproven through data, accounting for variables. In programming, variables are literally variables.

If we know exactly what our code will do, we have the flexibility to use it in different circumstances. That may sound tautological, but the confidence we know “exactly” what it will do comes through tests, not an internal sense of “I know what that function does." I would argue most bugs arise through either typos or incorrect assumptions. Most of the typo bugs are caught before release. Most of the insidious bugs that take forever to debug are because someone made an assumption along the way that you have to find and correct.

If all functions perform as we expect, integration issues are drastically reduced. Good unit testing reduces the amount of integration and regression testing you have to do. Maintenance overall becomes easier because we have solid bulwarks of functionality, rather than needing to reason through all possible eventualities fresh every time (or worse, assuming).

I’m not a huge believer in code coverage as a benchmark for quality. I think it can be helpful to have a minimal coverage requirement as you’re starting to remind yourself to write tests, but 100% coverage means absolutely nothing on its own. Quality is much more important than quantity when it comes to testing, especially that you’re testing the right things.

Keeping it simple also relates to abstractions. Code is a series of abstractions (unless you’re writing assembly, in which case, vaya con Dios), but I’m referring specifically to abstractions in the codebase that you write. The cardinal sin of object-oriented programming is a simple rule: “Don’t Repeat Yourself.” It’s not … bad advice, but neither is it a simple panacea we could automate away with, say, a linter or a formatter (or, god forbid, AI).

DRY is overemphasized, possibly because it’s such an easy heuristic to apply. “Hey this looks like other code” is easy to see at a glance, and if you just have an automatic reaction of “I shouldn’t repeat myself ever,” you’ll automatically push that logic up to single method that can be used in multiple places.

But deduplication requires an abstraction. In most cases, you’re not performing exactly the same logic in two places, but two minor variations (or the same logic on two different types of objects). Those variations then require you to include a parameter, to account for a slight branch.

Having that abstracted method hinders comprehensibility. Even if it’s easier/faster to read a one-line reference to the abstracted method, the actual logic being performed now happens out-of-sight.

I am much less concerned with duplication of code than I am making sure we find the right abstraction. Thus, I want to propose a different model for thinking about repetition, two rules  (because again, simpler != terse) to replace the one DRY rule: we’ll call it the Concilio Corollary to the DRY rule, or the damp dyad.

1. Don’t repeat yourself repeating yourself

2. The wrong abstraction will cost you more than repetition

DRYRY is a little tongue-in-cheek, but essentially don’t worry about trying to find an abstraction until you’ve implemented similar logic at least three times. Twice is a coincidence, three times is a pattern. Once you’ve seen the code being used in three different places, you now have the context to know whether it’s a) actually doing the same work, and b) how to abstract it to work in different scenarios.

If you find yourself adding a parameter that changes the flow of logic in each scenario, it’s probably more correct to abstract only those small parts that are the same, and implement the particular logic in the context it’s being used. That’s how we find the right abstraction.

All of this is important because existing code has inertia. This is relevant whether you’re a more senior developer or just starting out in your career.

Those with less experience tend to be terrified to change existing code, and understandably so. That code already exists, it’s doing a job. Even if you’re going in to fix a bug, presumably that code was working well enough when it was written that no one noticed it. And no one wants to be the one to create an error, so the existing code is treated as something close to sacrosanct.

For more experienced developers, know that when you’re writing code you’re creating the building blocks of the application. You’re setting the patterns that other developers later will try to implement, because it’s “correct” and becomes that codebase’s style. Heck, that’s literally the predictability maxim - we want to it look similar when it does similar things. But that means if you’re writing the wrong abstraction in one place, its impact may not be limited to that single area.

And when a new case arises, the next developer has to decide (without the context of the person who originally wrote it) whether to modify the existing abstraction, or create a new one. But the old one is “correct” (again, in that exists), so it’s safer to just use that one. Or, worst case, use it as a template to create a new abstraction. In either case, a new paradigm is being created that needs to be tested and raises the overhead on maintenance, because now we have a separate logic branch.

Those are the big topics I wanted to hit. The rest of these recommendations are important, but lack an overall theme. The biggest of these I want to discuss is commenting.

Naming should be used so we know the “what” of code, comments should be used so we know the “why.” I am not referring to automated comments here (e.g., explanations for input parameters in the like in JSDoc), but rather qualitative comments. I would argue that, currently, most existing comments I see would be superfluous if proper naming conventions were used.

What I want to see in a comment is why a particular variable is a particular value, when it’s not clear from the existing context.

⛔ Don't explain the what

const SOCIAL_MEDIA_CHARACTER_COUNT = 116;
// shortens title for social media sharing
export const getSocialShareText = (post: BlogPost) => {
	if (post.title.length =< SOCIAL_MEDIA_CHARACTER_COUNT) {
		return post.title;
	} else {
		return post.title.substr(0,SOCIAL_MEDIA_CHARACTER_COUNT);
	}
}

This a pretty typical example of what I see comments used for. We’ve used naming properly (the method gets the social share text, the constant is the character count we use for social media posts)j, so the comment “shortens title for social media sharing” is superfluous.

This method provides the social media content. The piece of information I don’t have about this code that I would like, both for comprehensibility and maintainability, is why the character count is 116.

The answer is that Twitter used to be the social media service with the shortest content length, 140 characters. Except that since we’re developing an app, we’re always including a URL, for which Twitter automatically generates a shortlink that takes up 23 characters (+ 1 for the space between content and link). 140-23-1 = 116.

That context does not exist within the application, and it’s not under our control. So we should include it in a comment, so that if that number changes (or something else becomes popular but has a shorter length limit, or we stop worrying about Twitter entirely), we know both from reading the code what this does, and it puts a signpost with the word “Twitter” in the comment so it can be found if we just do a search.

✅ Explain the "why"

// Twitter has shortest character limit (140); URL shortener is always 23 + space
const SOCIAL_MEDIA_CHARACTER_COUNT = 116;
export const getSocialShareText = (post: BlogPost) => {
	if (post.title.length =< SOCIAL_MEDIA_CHARACTER_COUNT) {
		return post.title + ' ' + post.url;
	} else {
		return post.title.substr(0,SOCIAL_MEDIA_CHARACTER_COUNT) + ' ' + post.url;
	}
}

The other thing to keep in mind about comments is that they’re a dependency just as much as code. If we do update that character count, we also need to update the comment explaining it, otherwise we’ve actively corrupted the context for the next person who has to make a change.

I used to say “never use ternaries,” but I’ve come around a bit. I now believe ternaries should be used only declaratively, with proper formatting.

✅ Use declarative ternaries, with formatting

const title = (postRequest['title']) 
			? postRequest['title'] 
			: '';

const title = postRequest['title'] || '';

Ternaries are short, concise, and difficult to reason about if they’re too complicated. When I say “declarative” ternaries, I mean “the value of a variable is one of two options, dependent upon a single condition.”

If you need to test multiple conditions, or if you have more than one variable changing as a result of a condition or set of conditions, don’t use ternaries. Use regular if-else statements. It’s easier to read and comprehend, and it’s easier to make changes down the road (more likely if already have multiple conditions or states).

And never nest ternaries.

The last bit is around testing, specifically standard library functions. A standard library function is one that comes packaged in with the programming language you’re using - think Math.round() for Javascript, or the above substring method on strings str.substr(0,3).

As a rule, you should not test the functionality of code you have no control over - if Chrome is shipping a bad Math.round(), there isn’t anything you can do about it (plus, if you go down that rabbit hole long enough you’ll eventually have to test that the heat death of the universe hasn’t yet happened). Standard library functions fit that description.

But sometimes you do you want to test a method that only uses standard library functionality - the reason is not that you’re testing that functionality, but rather that you’re arriving at the desired result.

We’ll use the social media text as the example. I will always assume substring is working properly until I get user reports, and even then the most I would do is forward them along. What I want to test for is the length of the string that is returned - does it meet my requirements (under 116)? I’m not testing the functionality, I’m including a flag to myself and future developers that this is the maximum length and, if someone modifies the functionality of the method, it should be flagged.

describe('getSocialMediaText restricts to Twitter length', ()=> {
	it('when title is less than length', () => {
		expect(getSocialMediaText(MockPostShortTitle).length =< 116)
	}),
	it('when the title is more than length', () => {
		expect(getSocialMediaText(MockPostLongTitle).length =< 116)
	})
});

If we were testing functionality, I would call the same constant in my test, because that’s what’s being used internally. But because I’m testing outcomes, I use an independent value. If someone changes the length without changing the test, they’ll get notified. They can at that point change the value used in the test, too, but the test has served its purpose - it notified someone when they violated my “why.”

TL;DR

• Focus on specific aspects of code quality

  • Comprehensible, Predictable, Maintainable

• Name stuff properly

  • Clarity over concision and wit

• Keep things simple

  • One module, one class, one method, one variable: One job

• Write tests

  • The only way to confidently modify or reuse code is be assured of what it does

• Remember the damp dyad

  • Don’t repeat yourself repeating yourself

  • The wrong abstraction costs more than repetition

• Comments should explain "why"

  • Provide context for the next person (let naming focus on “what”)

Solutions come in all sizes. The problem in tech (and many other industries, I presume) is that our processes and workflows are structured in such a way that the solutions for a given problem tend to be clustered around the smaller side of the scale.

Consider any given bug. Reported (hopefully) by your QA team or, worst-case, by a customer in production, it points out a specific issue. You, the developer, are tasked with devising a solution. Now, in most shops you’ll be given the opportunity to work out the root cause, ensuring that whatever change you make will a) actually fix the problem, and b) not cause any other immediate problems.

And that makes sense, for the most part. Small issues have small solutions. The problem is when you don’t step back and take a bigger-picture view of the situation - do all of these disparate problems actually stem from a particular source? Very often, developers are not only encouraged but actually mandated to stick to whatever story they’re on, for fear of going out of scope.

While that might make sense from a top-down control perspective, that style of thinking tends to permeate a lot of the other work that gets done, even up to larger-scale issues. Diversity is left to HR, or to a diversity committee, to take care of. In many cases, how and where to include AI in an application is left up to individual departments or teams. Remote work, a topic extremely divisive of late, is being eliminated or limited left up to “manager discretion” rather than actually looking at the benefits and harms that are associated with it. A cause extremely close to my heart, accessibility, is frequently treated as an add-on or left up to a handful of specialists to implement (or, worse, a third-party plugin).

These things not only don’t have to, they shouldn’t be left up to small groups to implement or reason through. They should be baked-in to how your organization makes decisions, builds software and interacts with its people.

You need a holistic approach. I want to break these concepts out of silos. If we're looking at a RACI chart, everyone is responsible for DEIB and accessibility. Everyone should be consulted and accountable for decisions about AI and remote work.

Now, I have a confession. I'm pretty sure it's Steve Jobs’ Second Law of Product that any time you think you have an insight, you have to give it a fancy name. I am guilty of this as well. 

I use the term “holistic tech” to talk about the convergence of these ideas. A lot of the specific things I'm talking about can be found in other systems or methodologies; I'm just trying to pull all the threads together so we can hopefully weave something useful about it. In the same way that responsive design was concerned with making sure you could use a product across all screen sizes, I want to make sure that (and here's the subtitle) tech works for everybody.

I'm also gonna borrow some concepts from universal design. Universal design is the concept that, "the design and composition of an environment so that it can be accessed, understood and used to the greatest extent possible by all people regardless of their age, size, ability or disability."

And last, we'll also fold in some concepts of human-centered design. This, in a nutshell, is thinking beyond your optimal user story. Eric Meyer calls them "stress cases,"  as opposed to edge cases, where you consider the emotional, physical and mental state of your user, rather just concerning yourself with the state of your application. 

But all of these, as implied with the word "design," are focused primarily on product creation. And while I do want to incorporate that, it's a part of how we work.

Basically, this whole idea boils down to a single word

EMPATHY

It's about seeing other people as, well, people.  

And it's applicable up and down your company stack. It applies to your employees, your boss, your monetization strategy (specifically, not using dark patterns), and it's especially about your communication, both within your organization and with your users.

As for product design, we'll start with accessibility.

Very broadly, accessibility is concerned with making sure that everyone can access your content and product. On the web side of things, this typically is accomplished by trying to adhere to the Web Content Access Guidelines, or WCAG. 

WCAG has four basic principles:

• The first is that content should be perceivable, which relates to multi-sensory content and interfaces. Essentially, you should still be able to access the fundamental value of the content even if you cannot engage with its primary medium; the common examples here are alt text for images or captions for videos.

• The second principle is operable: Users must be able to operate user interface controls in multiple modalities. The most common example of this is keyboard navigability; there are several requirements around people being able to video controls or manipulate modals without using the mouse (or touch).

• The third principle is understandable: Text needs to be readable and understandable, and user interface elements should behave in predictable ways. Headers should always act like headers.

• The last principle is robustness, which amounts to future-proofing. Make sure you adhere to the specs so that future products that are trying to parse your content know they can do so in a coherent manner.

Now the interesting thing is, I don't think many people would object to those principles in, well, principle. They seem pretty common-sensical? "I want people to be able to access my content" is a fairly unobjectionable statement. The problem is that most organizations don't have a good sense for accessibility yet, so the projects are designed and budgeted without the specific accessibility implementations. Then, when it gets brought up, making the change would be "too expensive," or it would "take too long."

"And besides, it's an insignificant part of our market anyway." I cannot tell you how many times I've heard this argument. Whether it's an intranet ("we don't have that many disabled people working here") or an internal training video (“there aren’t that many blind workers”) or a consumer-facing product ("we're willing to live without that tiny part of the market"), there's a sense that accessibility is only for a very small subset of the population.

My favorite group accessibility experiment is to ask people to raise their hand if they use an accommodation.

Then, I ask them to raise a hand if they wear glasses, contacts, or a hearing aid.Or if they don't keep your monitor at full resolution ("less space," on Macs). Or if they ever change their  browser's or IDE's zoom level.

Those are all accessibility accommodations.

Because the truth of the matter is, we're all just temporarily abled. I don’t ask for hands on this one don't, but I’ll often ask if anyone’s ever bought something on eBay while drunk. Formally speaking, you are technically operating with a cognitive impairment when you bought that giant taco blanket on Amazon. And I'm willing to bet your fine motor skills weren't quite up their usual par, either.

Or maybe you sprained your wrist, or broke a finger. That's a loss of fine motor control that's going to make it more difficult to operate the mouse, even if only for a few weeks. Or how about any kind of injury or chronic pain that makes it painful to sit in a chair for long periods? Willing to bet after 4 hours you're not thinking as clearly or as quickly as you were during hour 1.

Some of these things, like neurodivergence or vision impairment or being paralyzed, can be permanent conditions. But just as many of them aren't. And it's important to keep that in mind, because even if your ideal user story is a 34-year-old soccer mom, chances are she's going to have some sort of cognitive impairment (lack of sleep, stress about kids) or processing difference (trying to juggle multiple things at the same time) or fine motor skills (trying to use your mobile app on the sidelines during December) at some point. So ignoring accessibility doesn’t just disenfranchise the “small” portion of your users who are visibly permanently disabled, it's making things more difficult for potentially all of your users at some point or another.

And as it turns out, adding accessibility features can actually grow your overall market share.

Imagine your first day at NewTube, the hottest new video app on the market. We're looking to change the world … by letting people upload and watch videos. I don’t know, venture capital! Anyway, the number of humans on the internet is 5.19 billion, so that’s our addressable market. We don’t need the microscopic share that would come from adding accessibility features.

Or do we?

Standard accessibility features for videos include text transcripts of the words spoken aloud in the video. The primary intention behind these is to ensure that those with hearing impairments can still understand what’s going on in the video. In a past job, proper captions cost somewhere in the range of $20+ per minute of video, though some products such as YouTube now have AI autocaptioning that’s getting pretty good. 

Another standard feature is an audio description track (and transcript). This is sort of like alt text for video –  it describes the images that are being shown on the screen, in order to make that information comprehensible to someone with visual impairments. 

My favorite example of this is the end scene from the movie Titanic. As a transcript, it looks like this:

[ music swells ]

ROSE: Uh!

[ splash ]

Audio description, on the other hand, would look something like this:

Present-day Rose walks to the bow of the research ship, which is deserted. Deep in thought, she climbs the railing and stares down at the water where the Titanic rests below. She opens one hand to reveal the Heart of the Ocean diamond. We flash back to 17-year-old Rose standing on the deck of the Carpathia, digging her hands into Cal’s overcoat and finding the diamond. Present day Rose shakes her head and, with a small gasp, sends the diamond to rest where it should have been some 80 years earlier.

I took some poetic license there, but that’s kind of the point of audio description – you’re not a court reporter transcribing what’s being said, you’re trying to convey the emotion and the story for those who can’t see the pictures. The transcript part isn’t technically a requirement, but since you typically have to write down the script for the AD track anyway, it tends to be included. To my knowledge, no one’s managed to get AI to do this work for them in any usable fashion.

Lastly, we have keyboard navigability. Being able to interact with and control the site just using a keyboard makes it easy for those without fine motor control (or who use screen readers) to easily find their way around.

Three features/feature sets. The first two are pretty expensive - we’ve either got to pay for or develop an AI service to write the transcriptions, or we have to make sure they’re available. Audio Descriptions are going to be a cost to us, regardless, and not a cheap one. Keyboard navigability could be built-in to the product, but it would be faster if we could just throw everything together in React and not have to worry about it. 

How much of an impact could it have on our audience?

Well, though only 2-3 children out of 1000 are born with hearing impairment, by age 18 the percentage of Americans who complain of at least partial hearing loss rises to about 15%. So if we don’t have captions, we’d better hope all our videos are Fail compilations, or we’re going to see some steep drop-offs.

When it comes to vision, it’s even worse. Approximately a billion people in the world have a vision impairment that was not prevented or has not been addressed. Even assuming significant overlap with the hearing impairment group, we’ll use 750,000,000, for a total of 10.8 percent.

And for inability to use a mouse, we’ll look at “overall prevalence of adults with a month of musculoskeletal pain related to a repetitive stress injury,” which isn’t nearly a large enough category to include everyone who might be navigating by keyboard, but is at 4%.

Which leaves us 70% of our addressable market, or 3.63 billion.

Now obviously these numbers are not exact. We’re very back-of-the-napkin here, but I would also argue that a real-world scenario could just as easily see our percentages of accommodation-seekers go up as down. The number of temporary cases of all of these items, the fact that first-world countries have higher prevalence of RSI (though much better numbers for vision impairment) mean that this 70% number is probably not as far away from reality as we think.

And even beyond people who need those accommodations, what about those who simply want them? 

My best friend watches TV with the captions on all the time because it’s easier for her to follow along, and she’s not alone. Netflix says 40% of global users watch with captions, to say nothing of public exhibitions like bars (where it’s often not legally permissible to have the sound on). 

Transcripts/audio descriptions are often HUGE boons to SEO, because you’re capturing all your content in a written, easily search-indexable format. 

And presumably you’ve used a video app on a TV. The app has already been designed to be used with directional arrows and an OK button - why not extend that to the desktop? You’ll notice the remote’s functionality is a subset of a keyboard, not a mouse. Boom, keyboard navigation.

So, to recap accessibility: Good for disabled users. Good for abled users. Good for business. And that’s the thing, taking a holistic approach to how we do tech should actually make everyone better off. It is the rising tide.

But let’s talk about the looming wave that overshadows us all. I speak, of course, of artificial intelligence. In the same way that software ate the world 15 years ago, and Bitcoin was going to replace all our dollars, artificial intelligence is going to eat all our software and all the dollars we software developers used to get paid. 

I want to make clear up front that I am not an AI doomsayer. I don’t think we’re (necessarily) going to get Skynetted, and if we are it’s certainly not going to be ChatGPT. Artificial intelligence in its current form is not going to enslave us, but I do think large swaths of the population will become beholden to it – just not in the same way. 

Similar to how algorithms were used in the 90s and 2000s to replace human decision-making, I think AI is going to be (ab)used in the same manner. We’ve all called in to a customer support line only to find that the human on the other end is little more than a conduit between “the system” and us, and the person can’t do anything more to affect the outcome than we can. 

With AI, we’re just going to skip the pretense of the human and have the AI decipher what it thinks you said, attempt remedies within the limits of what it’s been programmed to allow, and then disconnect you. No humans (or, likely, actual support) involved.

Is that the worst? Maybe not in all cases. But it’s also, in a lot of cases, going to allow these organizations to skip what should be important work and just let the AI make decisions. I’m much less concerned about SkyNet than I am the Paperclip Maximizer.

The paperclip maximizer is a thought experiment proffered by Nick Bostrom in 2003. He postulated that an AI given a single instruction, “Make as many paperclips as possible,” would/should end with the destruction of the entire earth and all human life. The AI is not given any boundaries, and humans might switch the machine off (thus limiting the number of paperclips), so the AI will eventually eliminate humans. But even if the AI thinks us benign, at some point the AI consumes all matter on the earth aside from humans, and we are just so full of wonderfully bendable atoms that could be used for more paperclips.

The “thought processes” of generative AIs, as currently constructed, are inherently unknowable. We know the inputs, and we can see the outputs when we put in a prompt, but we can’t know what they’re going to say - that’s where the special sauce “thinking” comes in. We try to control this by introducing parameters, or guidelines, to those prompts to keep them in line.

And I know you might think, “Well, we’ll tell it not to harm humans. Or animals. Or disrupt the existing socio-political order. Or …” And that’s actually a separate angle to attack this problem - humans not giving the proper parameters. At a certain point though, if you have to control for the entire world and its infinite varieties of issues, isn’t it easier to just do the work yourself? We’ve already got a lackluster track record in regard to putting reliable guardrails around AI, as the Bing Image Generator’s output so thoughtfully proves. 

One of the things computer nerds love to do more than anything is break new tech, and image generators are no exception. When it introduced a new image generation tool a while back, though Bing did restrict uses of the phrase “9/11" or “September 11,” it still allowed for image generations of “Spongebob flying an airliner into New York in 2000.” And of course, the most prominent image of New York in 2000 is likely going to include the World Trade Center.

Sure, Spongebob doing 9/11 is a brand hit to Nickelodeon and insulting to the victims’ families. But this is showing both failures - despite Bing’s overwhelming image consciousness that should have been baked into a model, the model thought it more important to generate this image than to not. And, separately, Bing failed to put proper safeguards into the system. 

So yes, the paperclips are a hyperbolic hypothetical, but if there’s one thing that capitalism has taught us it’s that there are companies out there who care more about the next dollar than anything else.

Businesses large and small make decisions based on weighing costs versus expected benefits of a given option all the time. Famously, with the Ford Pinto, one of the analyses Ford conducted cited the overall cost of redesigning fuel safety systems vs. the general cost to society of the fatal car crashes that might be spared. Because, to Ford, individual deaths were not thought of as particularly tragic. They were just numbers. It does not seem unreasonable to assume AI systems will be misused by those who are unscrupulous in addition to those who are just oblivious. 

In accessibility, most people think the cost of not being accessible is “well, how likely are we to get sued?” ignoring the benefits of people using the product more. With AI, this short-sighted calculus can come into play where “Oh, we’ll let the AI do it, and not have to pay a person!” Except, as we’ve pointed out, the AI probably isn’t very good and the cost comes in consumer goodwill.

And this doesn’t even touch things like source data bias, which is a huge issue in resume-reviewing AIs (whose datasets will cause the AI to be more likely to select for existing employees, exacerbating skewed hiring trends) and predictive policing algorithms (which exacerbate existing crime biases). 

Don’t forget you can now convincingly generate human-sounding responses in astroturfing campaigns or review spoofing, or empower scammers previously held back by non-native English suddenly sounding like every corporate communication (because AI’s probably writing those communiques, too).

Remember the part where I said I’m not an AI doomsayer? I’m really not! I think AI can be used in a lot of unique and interesting applications to make things better. We just need to be more judicious about how we employ it, is all. 

For example, in the medical field, there are numerous AI experiments around trying to find tumors from body scans; the AI is not notifying these people on its own, there are doctors who review flagged scans for closer examination. Or in drug trials, companies are using AI to imagine new shapes of proteins that will then have lots of trials and study before they’re ever put in a test subject. 

Using AI to generate advice that is then examined by humans for robustness is a great application of the tool. And sure, if Amazon wants to use AI to suggest product recommendations, I guess go ahead. It can’t be any worse than its current system of, “Oh, you bought a refrigerator? I bet you also want to buy several more.”

But that “generation” word is a sticking point for me. To the point of the job applicant winnowing, I have no problem with using quantitative questions to weed out applicants (do you have x years of experience, boolean can you work in the US), but I would hesitate to let a black-box system make decisions even as small as who should be considered for hiring based on inherently unknowable qualifications (as would be the case with the application of actual AI versus just algorithmic sifting).

And finally, just limit the use of generated content in general. Reaching back into my accessibility bag for a minute, there’s a class of images that per spec don’t need alt text: Images that are “purely” decorative and not conveying information. The question I always ask in such cases is: If the image is really providing no value to the user, do you really need it?

The same would go for unedited generated content. If you’re sending a communication that can be wholly generated by a computer, do you really need to send it? We’re taking the idea of “this meeting could have been an email” even further down the stack: Could that email in fact just be a Slack message, or better yet a reaction emoji? Just because you can expand your one-sentence idea more easily with AI doesn’t you have to or even should.

There’s likely a place for generated content, but it’s not anywhere near where we’re using it for now, with AI-generated “news” articles or advertising campaigns. It’s like when we just tried to add accessibility “with a button” - you cannot just throw this stuff out there and hope it’s good enough.

And I would hope it would go without saying, but please don’t replace therapists or lawyers or any other human who considers ethics, empathy, common sense or other essentially human traits with AI.

This is along the same lines as “generate advice not decisions,” - if you need to talk to the AI in order to be comfortable sharing things with a live person, that makes total sense. But don’t use the AI as a 1:1 replacement for talking to a person, or getting legal advice.

AI recap: Good for advice, not decisions. Good for assisting people, not replacing them (it’s a tool, not the mechanic). It can be good for business.

Now, I think at this point you can pretty much guess what I’m gonna say about remote work. And that’s good! Both because this is already long enough and because “holistic tech” is supposed to be a framework, not just specific actionable items. 

Remote work, of course, is the idea that you need not be physically present in a building in order to perform a job. Hybrid work is a mix of remote work with some time spent in the office. I’m not gonna try sell you hard on either option - but I will note that employees prefer flexibility and employers tend to enjoy the larger talent pool. But mostly, I want to talk about how to set up your organization for success in the event you choose one of them.

One of the issues when you have some people in the office and others who aren’t is the sense that employees in the office are prioritized above those who are remote. Some of this is understandable – if the company wants to incentivize people to come into the office by offering, for example, catered lunches once a week or something, I wouldn’t see that as something that those who aren’t attending are missing out on …. Unless they were hired as fully remote.

In my case, for example, company HQ is in Chicago; I live in Phoenix, Arizona. I was hired fully remote, and it would feel to me like I were a lesser class of employee if those in the Chicago area were regularly incentivized with free lunches when there’s no pragmatic way for me to partake. Luckily, our office uses a system where everyone gets the same amount of delivery credit when we have whole-office lunches, which allows all of us to feel included. 

Beyond incentives, though, is the actual work being done, and this is where I think some teams struggle. Especially when it comes to meetings, the experience of the remote attendee is often an afterthought. This can take the forms of whiteboarding (literally writing on a whiteboard off-camera in the room), crosstalk or side discussions that aren’t in listening range of the microphone, or showing something on a screen physically that’s not present virtually.

It’s not just “don’t punish your remote team members for being remote,” you’re actually hurting the organization as a whole. Presumably every member of the team was hired with an eye to what they can bring to the table; excluding them, or not giving them the full information, hurts everyone involved. 

And technological solutions for remote workers will benefit in-person workers as well! Talking into the microphone during meetings can help someone with cochlear implants hear better in the room just as much as it’ll help me sitting in my garage office 1200 miles away. Same goes for whiteboarding - having a Google Jam (is that where they’re called anymore? Bring back the Wave!) on their screen means my wife can actually follow along; if they have to read a whiteboard from even 14 feet away, they’ll lose track of what’s going on in the meeting.

Taking the time to plan for how the remote attendee’s experience helps everyone, and it’s not terribly difficult to do. You can even see it for yourself by simply attending the meeting from another room to give you perspective and help troubleshoot any issues. Part and parcel of this, of course, is investing in the tools necessary to make sure everyone can interact and collaborate on the same level.

It’s not all about managers/employers, though! Remote employees tend to think that remote work is just like being in the office, only they don’t have the commute. And while that’s true to some extent, there’s another crucial aspect that many of them are missing: Communication. 

You have to communicate early and often when you’re remote for the simple reason that no one can come check up on you. No one can look over your shoulder to see if you’re struggling, no one knows intuitively what your workload looks like if you’re overloaded. Similarly, you don’t know what impacts your coworker’s commit is going to have unless you ask them. There are any number of tools and video sharing apps and all that, but the upshot is you actually have to make focused efforts to use them to make sure everyone’s rowing in the same direction.

Remote work: good for employees, good for employers. Good for business.

Finally, let’s talk diversity. Commonly abbreviated DEI, or DEIB, diversity, equity, inclusion and belonging has sort of morphed from “let’s make sure our workforce looks diverse” to “let’s make sure people of different backgrounds feel like they have a place here.”

And that’s because DEIB should be a culture, not an initiative. At the start, we talked about silos vs. intersectionality. This might start with a one-off committee, or an exec hire, but true DEIB is about your entire culture. Just like remote work can’t just be HR’s problem, and AI decisions shouldn’t be made solely by the finance team, DEIB needs to come from the entire organization.

I actually like the addition of the B to DEI because Belonging is a pretty good shorthand for what we’ve been discussing throughout. People who are temporarily or permanently disabled are provided the accommodations they need to succeed and thrive; programmers aren’t worried AI is going to be used to replace them, but instead given to them as a tool to increase their productivity. Remote workers feel like the company values them even in a different state.

DEIB should encompass all those things, but it can’t be left up to just a committee or an exec or even a department to account for it. It all falls on all of us.

And I specifically don’t want to leave out the traditional aspects of diversity, especially in tech culture. Minorities of all kinds – women, nonbinary folks, other gender identities, those of different sexual orientations, non-white racial backgrounds – are underrepresented in our industry, and it’s important that we keep up the work required to make sure that everyone is given the same access and opportunities. 

It’s good for business, too! Having a diverse array of perspectives as you develop products will give you ideas or user stories or parameters a non-diverse group might never have thought of. We keep hearing stories about VR headsets that clearly weren’t designed for people with long hair, or facial recognition algorithms that only work for those with lighter skin tones. If your product serves everybody, your product will be used by more people. That’s basic math!

Recent court rulings have put a damper on what used to be the standard for diversity, a “quota” of either applicants or hires meeting certain criteria. And look, if your organization was hiring just to meet a metric, you didn’t have true diversity. Quotas don’t create a culture of inclusion, so them going away shouldn’t cause that culture to dissipate, either. Seek out diverse upstreams for your hiring pipeline, ensure you’re not just tapping the same sources. I promise you, that investment will provide a return.

Say it with me: DEIB is good for employees, good for employers, and it’s good for business.

TLDR: Have empathy. Make sure you consider all aspects of decisions before you make them, because very often taking the personhood of the other party into account is actually the best business move as well. 

And with all of these, please note that when I say these are good for employees, good for employers and, especially, “good for business” requires these ideas to be executed well. Doing it right means taking as many of these factors into account as you can. This is where holistic tech comes in as our overarching concept.

• When it comes to accessibility, the more you have, the more customers you can reach and the more options you give them. With a long lens, that tends to mean you wind up with more money. 

• When you’re considering applications for artificial intelligence, try to keep its influence to advice rather than making decisions, and consider the work that would need to be done in order to implement the solution without AI – if it’s not work you’re willing to do, is it worth doing at all, AI or no? 

• With remote work, you need to invest the time and resources to ensure your remote employees have the tools to communicate, while employees need to invest the time and energy to actually communicate.

• Finally, diversity and belonging are about your culture, not a committee or a quota. Invest in it, and you’ll reap rewards.

You know it's a good sign when the first thing I do after finishing an article is double-check whether the whole site is some sort of AI-generated spoof. The answer on this one was closer than you might like, but I do think it's genuine.

Jakob Nielsen, UX expert, has apparently gone and swallowed the AI hype by unhinging his jaw, if the overall subjects of his Substack are to be believed. And that's fine, people can have hobbies, but the man's opinions are now coming after one of my passions, accessibility, and that cannot stand.

Very broadly, Nielsen says that digital accessibility is a failure, and we should just wait for AI to solve everything.

This gif pretty much sums up my thoughts after a first, second and third re-read.

I got mad at literally the first actual sentence:

Accessibility has failed as a way to make computers usable for disabled users.

Nielsen's rubric is an undefined "high productivity when performing tasks" and whether the design is "pleasant" or "enjoyable" to use. He then states, without any evidence whatsoever, that the accessibility movement has been a failure.

Accessibility has not failed disabled users, it has enabled tens of millions of people to access content, services and applications they otherwise would not have. To say it is has failed is to not even make perfect the enemy of the good; it's to ignore all progress whatsoever.

I will be the first to stand in line to shout that we should be doing better; I am all for interfaces and technologies that help make content more accessible to more people. But this way of thinking skips over the array of accessible technology and innovations that have been developed that have made computers easier, faster and pleasant to use.

For a very easy example, look at audio description for video. Content that would have been completely inaccessible to someone with visual impairments (video with dialogue) can now be understood through the presentation of the same information in a different medium.

Or what about those with audio processing differences? They can use a similar technology (subtitles) to have the words that are being spoken aloud present on the video, so they more easily follow along.

There are literally hundreds, if not thousands of such ideas (small and large) that already exist and are making digital interfaces more accessible. Accessibility is by no means perfect, but it has succeeded already for untold millions of users.

The excuse

Nielsen tells us there are two reasons accessibility has failed: It's expensive, and it's doomed to create a substandard user experience. We'll just tackle the first part for now, as the second part is basically just a strawman to set up his AI evangelism.

Accessibility is too expensive for most companies to be able to afford everything that’s needed with the current, clumsy implementation.

This line of reasoning is absolute nonsense. For starters, this assumes that accessibility is something separate from the actual product or design itself. It's sort of like saying building a nav menu is too expensive for a company to afford - it's a feature of the product. If you don't have it, you don't have a product.

Now, it is true that remediating accessibility issues in existing products can be expensive, but the problem there is not the expense or difficulty in making accessible products, it's that it wasn't baked into the design before you started.

It's much more expensive to retrofit a building for earthquake safety after it's built, but we still require that skyscrapers built in California not wiggle too much. And if the builders complain about the expense, the proper response is, "Then don't build it."

If you take an accessible-first approach (much like mobile-first design), your costs are not appreciably larger than ignoring it outright. And considering it's a legal requirement for almost any public-facing entity in the US, Canada or EU, it is quite literally the cost of doing business.

A detour on alt text

As an aside, the above image is a good example of the difference between the usability approach and the accessibility approach to supporting disabled users. Many accessibility advocates would insist on an ALT text for the image, saying something like: “A stylized graphic with a bear in the center wearing a ranger hat. Above the bear, in large, rugged lettering, is the phrase "MAKE IT EASY." The background depicts a forest with several pine trees and a textured, vintage-looking sky. The artwork has a retro feel, reminiscent of mid-century national park posters, and uses a limited color palette consisting of shades of green, brown, orange, and white.” (This is the text I got from ChatGPT when I asked it to write an ALT text for this image.)

On the other hand, I don’t want to slow down a blind user with a screen reader blabbering through that word salad. Yes, I could — and should — edit ChatGPT’s ALT text to be shorter, but even after editing, a description of the appearance of an illustration won’t be useful for task performance. I prefer to stick with the caption that says I made a poster with the UX slogan “Keep It Simple.”

The point of alt text is to provide a written description of visual indicators. It does NOT require you to describe in painstaking detail all of the visual information of the image in question. It DOES require you to convey the same idea or feeling you were getting across with the image.

If, in the above case, all that is required is the slogan, then you should not include the image on the page. You are explicitly saying that it is unimportant. My version of the alt text would be, "A stylized woodcut of a bear in a ranger hat evoking National Park posters sits over top of text reading "Make it easy.""

Sorry your AI sucks at generating alt text. Maybe you shouldn't rely on it for accessibility because it true accessibility requires ascertaining intent and including context?

The easy fix

Lolz, no.

The "solution" Nielsen proposes should be no surprise: Just let AI do everything! Literally, in this case, he means "have the AI generate an entire user experience every time a user accesses your app," an ability he thinks is no more than five years away. You know, just like how for the past 8 years full level 5 automated driving is no more than 2-3 years away.

Basically, the AI is given full access to your "data and features" and then cobbles together an interface for you. You as the designer get to choose "the rules and heuristics" the AI will apply, but other than that you're out of luck.

This, to be frank, sounds terrible? The reason we have designers is to present information in a coherent and logical flow with a presentation that's pleasing to the eye.

The first step is the AI will be ... inferring? Guessing? Prompting you with a multiple choice quiz? Reading a preset list of disabilities that will be available to every "website" you visit?

It will then take that magic and somehow customize the layout to benefit you. Oddly, the two biggest issues that Nielsen writes about are font sizes and reading level; the first of which is already controllable in basically every text-based context (web, phone, computer), and the second of which requires corporations to take on faith that the AI can rewrite their content completely while maintaining any and all style and legal requirements. Not what I'd bet my company on, but sure.

But my biggest complaint about all of this is it fails the very thing Nielsen is claiming to solve: It's almost certainly going to be a "substandard user experience!" Because it won't be cohesive, there have literally been no thought into how it's presented to me. We as a collective internet society got fed up with social media filter bubbles after about 5 years of prolonged use, and now everything I interact with is going to try to be intensely personalized?

Note how we just flat-out ignore any privacy concerns. I'm sure AI will fix it!

I really don't hate AI

AI is moderately useful in some things, in specific cases, where humans can check the quality of its work. As I've noted previously, right now we have not come up with a single domain where AI seems to hit 100% of its quality markers.

But nobody's managed to push past that last 10% in any domain. It always requires a human touch to get it "right."

Maybe AI really will solve all of society's ills in one fell swoop. But instead of trying to pivot our entire society around that one (unlikely) possibility, how about we actually work to make things better now?

More on this nonsense article

Today I want to talk about data transfer objects, a software pattern you can use to keep your code better structured and metaphorically coherent.

I’ll define those terms a little better, but first I want to start with a conceptual analogy.

It is a simple truth that, no matter whether you focus on the frontend, the backend or the whole stack, everyone hates CSS.

I kid, but also, I don’t.

CSS is probably among the most reviled of technologies we have to use all the time. The syntax and structure of CSS seems almost intentionally designed to make it difficult to translate from concept to “code,” even simple things. Ask anyone who’s tried to center a div.

And there are all sorts of good historical reasons why CSS is the way it is, but most developers find it extremely frustrating to work with. It’s why we have libraries and frameworks like Tailwind. And Bulma. And Bootstrap. And Material. And all the other tools we use that try their hardest to make sure you never have to write actual while still reaping the benefits of a presentation layer separate from content.

And we welcome these tools, because it means you don’t need to understand the vagaries of CSS in order to get what you want.
It’s about developer experience, making it easier on developers to translate their ideas into code.

And in the same way we have tools that cajole CSS into giving us what we want, I want to talk about a pattern that allows you to not worry about anything other than your end goal when you’re building out the internals of your application. It’s a tool that can help you stay in the logical flow of your application, making it easier to puzzle through and communicate about the code you’re writing, both to yourself and others. I’m talking about DTOs.

DTOs

So what is a DTO? Very simply, a data transfer object is a pure, structured data object - that is, an object with properties but no methods. The entire point of the DTO is to make sure that you’re only sending or receiving exactly the data you need to accomplish a given function or task - no more, no less. And you can be assured that your data is exactly the right shape, because it adheres to a specific schema.

And as the “transfer” part of the name implies, a DTO is most useful when you’re transferring data between two points. The title refers to one of the more common exchanges, when you’re sending data between front- and back-end nodes, but there are lots of other scenarios where DTOs come in handy.

Sending just the right amount of data between modules within your application, or consuming data from different sources that use different schemas, are just some of those.

I will note there is literature that suggests the person who coined the term, Martin Fowler, believes that you should not have DTOs except when making remote calls. He’s entitled to his opinion (of which he has many), but I like to reuse concepts where appropriate for consistency and maintainability.

The DTO is one of my go-to patterns, and I regularly implement it for both internal and external use.
I’m also aware most people already know what pure data objects are. I’m not pretending we’re inventing the wheel here - the value comes in how they’re applied, systematically.

Advantages

1. For DTOs are a systematic approach to managing how your data flows through and between different parts of your application as well as external data stores.

2. Properly and consistently applied, DTOs can help you maintain what I call metaphorical coherence in your app. This is the idea that the names of objects in your code are the same names exposed on the user-facing side of your application.

   Most often, this comes up when we’re discussing domain language - that is, your subject-matter-specific terms (or jargon, as the case may be).

   I can’t tell you the number of times I’ve had to actively work out whether a class with the name of “post” refers to a blog entry, or the action of publishing an entry, or a location where someone is stationed. Or whether “class” refers to a template for object creation, a group of children, or one’s social credibility. DTOs can help you keep things organized in your head, and establish a common vernacular between engineering and sales and support and even end-users.

   It may not seem like much, but that level of clarity makes talking and reasoning about your application so much easier because you don’t have to jump through mental hoops to understand the specific concept you’re trying to reference.

3. DTOs also help increase type clarity. If you’re at a shop that writes Typescript with “any” as the type for everything, you have my sympathies, and also stop it. DTOs might be the tool you can wield to get your project to start to use proper typing, because you can define exactly what data’s coming into your application, as well as morphing it into whatever shape you need it to be on the other end.

4. Finally, DTOs can help you keep your code as modular as possible by narrowing down the data each section needs to work with. By avoiding tight coupling, we can both minimize side effects and better set up the code for potential reuse.

And, as a bonus mix of points two and four, when you integrated with an external source, DTOs can help you maintain your internal metaphors while still taking advantage of code or data external to your system.

To finish off our quick definition of terms, a reminder that PascalCase is where all the words are jammed together with the first letter of each word capitalized; camelCase is the same except the very first letter is lowercase; and snake case is all lowercase letters joined by underscores.

This is important for our first example.

Use-case 1: FE/BE naming conflicts

The first real-world use-case we’ll look at is what was printed on the box when you bought this talk. That is, when your backend and frontend don’t speak the same language, and have different customs they expect the other to adhere to.

Trying to jam them together is about as effective as when an American has trouble ordering food at a restaurant in Paris and compensates by yelling louder.

In this example, we have a PHP backend talking to a Typescript frontend.

I apologize for those who don’t know one or both languages. For what it’s worth, we’ll try to keep the code as simple as possible to follow, with little-to-no language-specific knowledge required. In good news, DTOs are entirely language agnostic, as we’ll see as we go along.

Backend

class User 
{
	public function __construct(
		public int $id,
		public string $full_name,
		public string $email_address,
		public string $avatar_url
	){}
}

Per PSR-12, which is the coding standard for PHP, class cases must be in PascalCase, method names must be implemented in camelCase. However, the guide “intentionally avoids any recommendation” as to styling for property names, instead just choosing “consistency.”

Very useful for a style guide!

As you can see, the project we’re working with uses snake case for its property names, to be consistent with its database structure.

Frontend

class User {
	userId: number;
	fullName: string;
	emailAddress: string;
	avatarImageUrl: string;

	load: (userId: number) => {/* load from DB */};
	save: () => {/* persist */};
}

But Typescript (for the most part, there’s not really an “official” style guide in the same manner but most your Google, your Microsofts, your Facebooks tend to agree) that you should be using camelCase for your variable names.

I realize this may sound nit-picky or like small potatoes to those of used to working as solo devs or on smaller teams, but as organizations scales up consistency and parallelism in your code is vital to making sure both that your code and data have good interoperability, as well as ensuring devs can be moved around without losing significant chunks of time simply to reteach themselves style.

Now, you can just choose one of those naming schemes to be consistent across the frontend and backend, and outright ignore one of the style standards.

Because now your project asking one set of your developers to context-switch specifically for this application. It also makes your code harder to share (unless you adopt this convention-breaking in your extended cinematic code universe). You’ve also probably killed a big rule in your linter, which you now have to customize in all implementations.

OR, we can just use DTOs.

Now, I don’t have a generic preference whether the DTO is implemented on the front- or the back-end — that determination has more to do with your architecture and organizational structure than anything else.

Who owns the contracts in your backend/frontend exchange is probably going to be the biggest determiner - whichever side controls it, the other is probably writing the DTO. Though if you’re consuming an external data source, you’re going to be writing that DTO on the frontend.

Where possible, I prefer to send the cleanest, least amount of data required from my backend, so for our first example we’ll start there. Because we’re writing the DTO in the backend, the data we send needs to conform to the schema the frontend expects - in this instance, Typescript’s camel case.

Backend

class UserDTO
{
	public function __construct(
		public int $userId,
		public string $fullName,
		public string $emailAddress,
		public string $avatarImageUrl
	) {}
}

That was easy, right? We just create a data object that uses the naming conventions we’re adopting for sharing data. But of course, we have to get our User model into the DTO. This brings me to the second aspect of DTOs, the secret sauce - the translators.

Translators

function user_to_user_dto(User $user): UserDTO
{
	return new UserDTO(
		$user->id,
		$user->full_name,
		$user->email_address,
		$user->avatar_url
	);
}

Very simply, a translator is the function (and it should be no more than one function per level of DTO) that takes your original, nonstandard data and jams it into the DTO format.

Translators get called (and DTOs are created) at points of ingress and egress. Whether that’s internal or external, the point at which a data exchange is made is when a translator is run and a DTO appears – which side of the exchange is up to your implementation.
You may also, as the next example shows, just want to include the translator as part of the DTO.

Using a static create method allows us to keep everything nice and contained, with a single call to the class.

class UserDTO
{
	public function __construct(
		public int $userId,
		public string $fullName,
		public string $emailAddress,
		public string $avatarImageUrl
	) {}
	
	public static function from_user(User $user): UserDTO
	{
		return new self(
			$user->id,
			$user->full_name,
			$user->email_address,
			$user->avatar_url
		);
	}
}
	
$userDto = UserDTO::from_user($user);

I should note we’re using extremely simplistic base models in these examples. Often, something as essential as the user model is going to have a number of different methods and properties that should never get exposed to the frontend.

While you could do all of this through customizing the serialization method for your object. I would consider that to be a distinction in implementation rather than strategy.

An additional benefit of going the separate DTO route is you now have an explicitly defined model for what the frontend should expect. Now, your FE/BE contract testing can use the definition rather than exposing or digging out the results of your serialization method.

So that’s a basic backend DTO - great for when you control the data that’s being exposed to one or potentially multiple clients, using a different data schema.

Please bear with me - I know this probably seems simplistic, but we’re about to get into the really useful stuff. We gotta lay the groundwork first.

Frontend

Let’s back up and talk about another case - when you don’t control the backend. Now, we need to write the DTO on the frontend.

First we have our original frontend user model.

class User {
	userId: number;
	fullName: string;
	emailAddress: string;
	avatarImageUrl: string;

	load: (userId: number) => {/* load from DB */};
	save: () => {/* persist */};
}

Here is the data we get from the backend, which I classify as a Response, for organizational purposes. This is to differentiate it from a Payload, which data you send to the API (which we’ll get into those later).

interface UserResponse {
	id: number;
	full_name: string;
	email_address: string;
	avatar_url: string;
}

You’ll note, again, because we don’t control the structure used by the backend, this response uses snake case.

So we need to define our DTO, and then translate from the response.

Translators

You’ll notice the DTO looks basically the same as when we did it on the backend.

interface UserDTO {
	userId: number;
	fullName: string;
	emailAddress: string;
	avatarImageUrl: string;
}

But it's in the translator you can now see some of the extra utility this pattern offers.

const translateUserResponseToUserDTO = 
	(response: UserResponse): UserDTO => ({
		userId: response.id,
		fullName: response.full_name,
		emailAddress: response.email_address,
		avatarImageUrl: response.avatar_url
});

When we translate the response, we can change the names of the parameters before they ever enter the frontend system. This allows us to maintain our metaphorical coherence within the application, and shield our frontend developers from old/bad/outdated/legacy code on the backend.

Another nice thing about using DTOs in the frontend, regardless of where they come from, is they provide us with a narrow data object we can use to pass to other areas of the application that don’t need to care about the methods of our user object.

DTOs work great in these cases because they allow you to remove the possibility of other modules causing unintended consequences.

Notice that while the User object has load and save methods, our DTO just has the properties. Any modules we pass our data object are literally incapable of propagating manipulations they might make, inadvertently or otherwise. Can’t make a save call if the object doesn’t have a save method.

Use-case 2: Metaphorically incompatible systems

For our second use-case, let’s talk real-world implementation. In this scenario, we want to join up two systems that, metaphorically, do not understand one another.

Magazine publisher

• Has custom backend system (magazines)

• Wants to explore new segment (books)

• Doesn’t want to build a whole new system

I worked with a client, let’s say they’re a magazine publisher. Magazines are a dying art, you understand, so they want to test the waters of publishing books.

But you can’t just build a whole new app and infrastructure for an untested new business model. Their custom backend system was set up to store data for magazines, but they wanted to explore the world of novels. I was asked them build out that Minimum Viable Product.

Existing structure

interface Author {
	name: string;
	bio: string;
}

interface Article {
	title: string;
	author: Author;
	content: string;
}
	
interface MagazineIssue {
	title: string;
	issueNo: number;
	month: number;
	year: number;
	articles: Article[];
}

This is the structure of the data expected by both the existing front- and back-ends. Because everything’s one word, we don’t even need to worry about incompatible casing.

Naive implementation
This new product requires performing a complete overhaul of the metaphor.

interface Author {
	name: string;
	bio: string;
}

interface Chapter {
	title: string;
	author: Author;
	content: string;
}

interface Book {
	title: string;
	issueNo: number;
	month: number;
	year: number;
	articles: Chapter[];
}

But we are necessarily limited by the backend structure as to how we can persist data.

If we just try to use the existing system as-is, but change the name of the interfaces, it’s going to present a huge mental overhead challenge for everyone in the product stack.

As a developer, you have to remember how all of these structures map together. Each chapter needs to have an author, because that’s the only place we have to store that data. Every book needs to have a month, and a number. But no authors - only chapters have authors.

So we could just use the data structures of the backend and remember what everything maps to. But that’s just asking for trouble down the road, especially when it comes time to onboard new developers. Now, instead of them just learning the system they’re working on, they essentially have to learn the old system as well.

Plus, if (as is certainly the goal) the transition is successful, now their frontend is written in the wrong metaphor, because it’s the wrong domain entirely. When the new backend gets written, we’re going to have to the exact same problem in the opposite direction.

I do want to take a moment to address what is probably obvious – yes, the correct decision would be to build out a small backend that can handle this, but I trust you’ll all believe me when I say that sometimes decisions get made for reasons other than “what makes the most sense for the application’s health or development team’s morale.”

And while you might think that find-and-replace (or IDE-assisted refactoring) will allow you to skirt this issue, please trust me that you’re going to catch 80-90% of cases and spend twice as much time fixing the rest as it would have to write the DTOs in the first place.

Plus, as in this case, your hierarchies don’t always match up properly.

What we ended up building was a DTO-based structure that allowed us to keep metaphorical coherence with books but still use the magazine schema.

Proper implementation

You’ll notice that while our DTO uses the same basic structures (Author, Parts of Work [chapter or article], Work as a Whole [book or magazine]), our hierarchies diverge. Whereas Books have one author, Magazines have none; only Articles do.

The author object is identical from response to DTO.

You’ll also notice we completely ignore properties we don’t care about in our system, like IssueNo.

How do we do this? Translators!

Translating the response

We pass the MagazineResponse in to the BookDTO translator, which then calls the Chapter and Author DTO translators as necessary.

export const translateMagazineResponseToAnthologyBookDTO = 
	(response: MagazineResponse): AnthologyBookDTO => {
		const chapters = (response.articles.length > 0) ? 
			response.articles.forEach((article) => 
				translateArticleResponseToChapterDTO(article)) : 
				[];
		const authors = [
			...new Set(
				chapters
					.filter((chapter) => chapter.author)
					.map((chapter) => chapter.author)
			)	
		];
		return {title: response.title, chapters, authors};
};

export const translateArticleResponseToChapterDTO = 
	(response: ArticleResponse): ChapterDTO => ({
		title: response.title,
		content: response.content,
		author: response.author
});

This is also the first time we’re using one of the really neat features of translators, which is the application of logic. Our first use is really basic, just checking if the Articles response is empty so we don’t try to run our translator against null. This is especially useful if your backend has optional properties, as using logic will be necessary to properly model your data.

But logic can also be used to (wait for it) transform your data when we need to.

Remember, in the magazine metaphor, articles have authors but magazine issues don’t. So when we’re storing book data, we’re going to use their schema by grabbing the author of the first article, if it exists, and assign it as the book’s author. Then, our chapters ignore the author entirely, because it’s not relevant in our domain of fiction books with a single author.

Because the author response is the same as the DTO, we don’t need a translation function. But we do have proper typing so that if either of them changes in the future, it should throw an error and we’ll know we have to go back and add a translation function.

The payload

Of course, this doesn’t do us any good unless we can persist the data to our backend. That’s where our payload translators come in - think of Payloads as DTOs for the anything external to the application.

interface AuthorPayload
	name: string;
	bio: string;
}

interface ArticlePayload {
	title: string;
	author: Author;
	content: string;
}

interface MagazineIssuePayload {
	title: string;
	issueNo: number;
	month: number;
	year: number;
	articles: ArticlePayload[];
}

For simplicity’s sake we’ll assume our payload structure is the same as our response structure. In the real world, you’d likely have some differences, but even if you don’t it’s important to keep them as separate types. No one wants to prematurely optimize, but keeping the response and payload types separate means a change to one of them will throw a type error if they’re no longer parallel, which you might not notice with a single type.

Translating the payload

export const translateBookDTOToMagazinePayload = 
	(book: BookDTO): MagazinePayload => ({
		title: book.title,
		articles: (book.chapters.length > 0) ? 
			book.chapters.forEach((chapter) => 
				translateChapterDTOToArticlePayload(chapter, book) : [],
		issueNo: 0,
		month: 0,
		year: 0,
});

export const translateChapterDTOToArticlePayload = 
	(chapter: ChapterDTO, book: BookDTO): ArticlePayload => ({
		title: chapter.title,
		author: book.author,
		content: chapter.content
}); 

Our translators can be flexible (because we’re the ones writing them), allowing us to pass objects up and down the stack as needed in order to supply the proper data.

Note that we’re just applying the author to every article, because a) there’s no harm in doing so, and b) the system like expects there be an author associated with every article, so we provide one. When we pull it into the frontend, though, we only care about the first article.

We also make sure to fill out the rest of the data structure we don’t care about so the backend accepts our request. There may be actual checks on those numbers, so we might have to use more realistic data, but since we don’t use it in our process, it’s just a question of specific implementation.

So, through the application of ingress and egress translators, we can successfully keep our metaphorical coherence on our frontend while persisting data properly to a backend not configured to the task. All while maintaining type safety. That’s pretty cool.

The single biggest thing I want to impart from this is the flexibility that DTOs offer us.

Use-case 3: Using the smallest amount of data required

When working with legacy systems, I often run into a mismatch of what the frontend expects and what the backend provides; typically, this results in the frontend being flooded an overabundance of data.

These huge data objects wind up getting passed around and used on the frontend because, for example, that’s what represents the user, even if you only need a few properties for any given use-case.

Or, conversely, we have the tiny amount of data we want to change, but the interface is set up expecting the entirety of the gigantic user object. So we wind up creating a big blob of nonsense data, complete with a bunch of null properties and only the specific ones we need filled in. It’s cumbersome and, worse, has to be maintained so that whenever any changes to the user model need to be propagated to your garbage ball, even if those changes don’t touch the data points you care about.

One way to eliminate the data blob is to use DTOs to narrowly define which data points a component or class needs in order to function. This is what I call minimizing touchpoints, referring to places in the codebase that need to be modified when the data structure changes.

In this scenario, we’re building a basic app and we want to display an avatar for a user. We need their name, a picture and a color for their frame.

const george = {
	id: 303;
	username: 'georgehernandez';
	groups: ['users', 'editor'],
	sites: ['https://site1.com'],
	imageLocation: '/assets/uploads/users/gh-133133.jpg';
	profile: {
		firstName: 'George';
		lastName: 'Hernandez';
		address1: '738 Evergreen Terrace';
		address2: '';
		city: 'Springfield';
		state: 'AX';
		country: 'USA';
		favoriteColor: '#1a325e';
	}
}

What we have is their user object, which contains a profile and groups and sites the user is assigned to, in addition to their address and other various info.

Quite obviously, this is a lot more data than we really need - all we care about are three data points.

class Avatar
{
	private imageUrl: string;
	private hexColor: string;
	private name: string;
	
	constructor(user: User)
	{
		this.hexColor = user.profile.favoriteColor:
		this.name = user.profile.firstName 
			+ ' ' 
			+ user.profile.lastName;
		this.imageUrl = user.imageLocation;
	}
}

This Avatar class works, technically speaking, but if I’m creating a fake user (say it’s a dating app and we need to make it look like more people are using than actually is the case), I now have to create a bunch of noise to accomplish my goal.

const lucy = {
	id: 0;
	username: '';
	groups: []; 
	sites: [];
	profile: {
		firstName: 'Lucy'; 
		lastName: 'Evans';
		address1: '';
		address2: '';
		city: '';
		state: '';
		country: '';
	}
	favoriteColor: '#027D01'
}

Even if I’m calling from a completely separate database and class, in order to instantiate an avatar I still need to provide the stubs for the User class.

Or we can use DTOs.

class Avatar
{
	private imageUrl: string;
	private hexColor: string;
	private name: string;
	
	constructor(dto: AvatarDTO)
	{
		this.hexColor = dto.hexColor:
		this.name = dto.name;
		this.imageUrl = dto.imageUrl;
	}
}

interface AvatarDTO {
	imageUrl: string;
	hexColor: string;
	name: string;
}

const translateUserToAvatarDTO = (user: User): AvatarDTO => ({
	name: [user.profile.firstName, user.profile.lastName].join(' '),
	imageUrl: user.imageLocation,
	hexColor: user.profile.favoriteColor
});

By now, the code should look pretty familiar to you. This pattern is really not that difficult once you start to use it - and, I’ll wager, a lot of you are already using it, just not overtly or systematically. The bonus to doing it in a thorough fashion is that refactoring becomes much easier - if the frontend or the backend changes, we have a single point from where the changes emanate, making them much easier to keep track of.

Flexibility

But there’s also flexibility. I got some pushback from implementing the AvatarDTO; after all, there were a bunch of cases already extant where people were passing the user profile, and they didn’t want to go find them. As much as I love clean data, I am a consultant; to assuage them, I modified the code so as to not require extra work (at least, at this juncture).

class Avatar
{
	private avatarData: AvatarDTO;
	
	constructor(user: User|null, dto?: AvatarDTO)
	{
		if (user) {
			this.avatarData = translateUserToAvatarDTO(user);
		} else if (dto) {
			this.avatarData = dto;
		}
	}
}

new Avatar(george);
new Avatar(null, {
	name: 'Lucy Evans',
	imageUrl: '/assets/uploads/users/le-319391.jpg', 
	hexColor: '#fc0006'
});

Instead of requiring the AvatarDTO, we still accept the user as the default argument, but you can also pass it null. That way I can pass my avatar DTO where I want to use it, but we take care of the conversion for them where the existing user data is passed in.

Use-case 4: Security

The last use-case I want to talk about is security. I assume some to most of you already get where I’m going with this, but DTOs can provide you with a rock-solid way to ensure you’re only sending data you’re intending to.

Somewhat in the news this month is the Spoutible API breach; if you’ve never heard of it, I’m not surprised. Spoutible a Twitter competitor, notable mostly for its appalling approach to API security.

I do encourage all of you to look this article up on troyhunt.com, as the specifics of what they were exposing are literally unbelievable.

{ 
  err_code: 0,
  status: 200,
  user: {
    id: 23333,
    username: "badwebsite",
    fname: "Brad",
    lname: "Website",
    about: "The collector of bad website security data",
    email: 'fake@account.com',
    ip_address: '0.0.0.0',
    verified_phone: '333-331-1233',
    gender: 'X',
    password: '$2y$10$r1/t9ckASGIXtRDeHPrH/e5bz5YIFabGAVpWYwIYDCsbmpxDZudYG'
  }
}

But for the sake of not spoiling all the good parts, I’ll just show you the first horrifying section of data. For authenticated users, the API appeared to be returning the entire user model - mundane stuff like id, username, a short user description, but also the password hash, verified phone number and gender.

Now, I hope it goes without saying that you should never be sending anything related to user passwords, whether plaintext or hash, from the server to the client. It’s very apparent when Spoutible was building its API that they didn’t consider what data was being returned for requests, merely that the data needed to do whatever task was required. So they were just returning the whole model.

If only they’d used DTOs! I’m not going to dig into the nitty-gritty of what it should have looked like, but I think you can imagine a much more secure response that could have been sent back to the client.

Summing up

If you get in the practice of building DTOs, it’s much easier to keep control of precisely what data is being sent. DTOs not only help keep things uniform and unsurprising on the frontend, they can also help you avoid nasty backend surprises as well.

To sum up our little chat today: DTOs are a great pattern to make sure you’re maintaining structured data as it passes between endpoints.

Different components only have to worry about exactly the data they need, which helps both decrease unintended consequences and decrease the amount of touchpoints in your code you need to deal with when your data structure changes. This, in turn, will help you maintain modular independence for your own code.

It also allows you to confidently write your frontend code in a metaphorically coherent fashion, making it easier to communicate and reason about.

And, you only need to conform your data structure to the backend’s requirements at the points of ingress and egress - Leaving you free to only concern your frontend code with your frontend requirements. You don’t have to be limited by the rigid confines backend’s data schema.

Finally, the regular use of DTOs can help put you in the mindset of vigilance in regard to what data you’re passing between services, without needing to worry that you’re exposing sensitive data due to the careless conjoining of model to API controller.

Honestly, I thought we were past this as an industry? But my experience at Developer Week 2024 showed me there's still a long way to go to overcoming sexism in tech.

And it came from the source I least expected; literally people who were at the conference trying to convince others to buy their product. People for whom connecting and educating is literally their job.

Time and again, both I (an engineer) and my nonbinary wife (a business analyst, at a different organization) found that the majority of the masculine-presenting folks at the booths on the expo floor were dismissive and disinterested, and usually patronizing.

It was especially ironic given one of the predominant themes of the conference was developer experience, and focusing on developer productivity. One of the key tenets of dx is listening to what developers have to say. These orgs failed. Horribly.

My wife even got asked when "your husband" would be stopping by.

I had thought it would go without saying, but female- and androgynous-presenting folk are both decision-makers in their own right as well as people with influence in companies large and small.

To organizations: Continuing to hire people who make sexist (and, frankly, stupid) judgments about who is worth talking to and what you should be conversing with them about is not only insulting, it's bad business. Founders: If you're the ones at your booth, educate yourselves. Fast.

I can tell you there are at least three different vendors who were providing services in areas we have professional needs around who absolutely will not get any consideration, simply because we don't want to deal with people like that. I don't assume the whole organization holds the same opinions as their representative; however, I can tell for a fact that such views are not disqualifying by that organization, and so I have no interest in dealing with them further.

Rather than call out the shitty orgs, I instead want to call out the orgs whose reps were engaging, knowledgable and overall pleasant to deal with. Both because those who do it right should be celebrated, and because in an attention economy any given (even negative) is unfortunately an overall net positive.

The guys at Convex were great, answering all my questions about their seemingly very solid and robust Typescript backend platform.

The folks at Umbraco gave great conference talks, plied attendees with cookies and talked with us at length about their platform and how we might use it. Even though I dislike dotNet, we are very interested in looking at them for our next CMS project.

The developer advocates at Couchbase were lovely and engaging, even if I disagree with Couchbase's overall stance on their ability to implement ACID.

The folks at the Incident.io booth were wonderful, and a good template for orgs trying to sell services: They brought along an engineering manager who uses their services, and could speak specifically to how to use them.

I want to give a shout-out to those folks, and to exhort organizations to do better in training those you put out as the voice of your brand. This is not hard. And it only benefits you to do so.

As part of my plan to spend more time bikeshedding building out my web presence than actually creating content, I wanted to build an iOS app that allowed me to share short snippets of text or photos to my blog. I've also always wanted to understand Swift generally and building an iOS app specifically, so it seemed like a nice little rabbit hole.

With the help of Swift UI Apprentice, getting a basic app that posted a content, headline and tags to my API wasn't super difficult (at least, it works in the simulator. I'm not putting it on my phone until it's more useful). I figured adding a share extension would be just as simple, with the real difficulty coming when it came time to posting the image to the server.

Boy was I wrong.

Apple's documentation on Share Extensions (as I think they're called? But honestly it's hard to tell) is laughably bad, almost entirely referring to sharing things out from your app, and even the correct shitty docs haven't been updated in it looks like 4+ years.

There are some useful posts out there, but most/all of them assume you're using UIKit. Since I don't trust Apple not to deprecate a framework they've clearly been dying to phase out for years, I wanted to stick to SwiftUI as much as I could. Plus, I don't reallllly want to learn two paradigms to do the same thing. I have enough different references to keep in my head switching between languages.

Thank god for Oluwadamisi Pikuda, writing on Medium. His post is an excellent place to get a good grasp on the subject, and I highly suggest visiting it if you're stuck. However, since Medium is a semi-paywalled content garden, I'm going to provide a cleanroom implementation here in case you cannot access it.

It's important to note that the extension you're creating is, from a storage and code perspective, a separate app. To the point that technically I think you could just publish a Share Extension, though I doubt Apple would allow it. That means if you want to share storage between your extension and your primary app, you'll need to create an App Group to share containers. If you want to share code, you'll need to create an embedded framework.

But once you have all that set up, you need to actually write the extension. Note that for this example we're only going to be dealing with text shared from another app, with a UI so you can modify it. You'll see where you can make modifications to work with other types.

You start by creating a new target (File -> New -> Target, then in the modal "Share Extension").

Once you fill out the info, this will create a new directory with a UIKit Storyboard file (MainInterface), ViewController and plist. We're not gonna use hardly any of this. Delete the Storyboard file. Then change your ViewController to use the UIViewController class. This is where we'll define what the user sees when content is shared. The plist is where we define what can be passed to our share extension.

There are only two functions we're concerned about in the ViewController — viewDidLoad() and close(). Close is going to be what closes the extension while viewDidLoad, which inits our code when the view is loaded into memory.

For close(), we just find the extensionContext and complete the request, which removes the view from memory.

viewDidLoad(), however, has to do more work. We call the super class function first, then we need to make sure we have access to the items that are been shared to us.

import SwiftUI

class ShareViewController: UIViewController {

    override func viewDidLoad() {
       super.viewDidLoad()
       
       // Ensure access to extensionItem and itemProvider
       guard
           let extensionItem = extensionContext?.inputItems.first as? NSExtensionItem,
           let itemProvider = extensionItem.attachments?.first else {
           self.close()
           return
       }
     }

   func close() {
       self.extensionContext?.completeRequest(returningItems: [], completionHandler: nil)
   }
}

Since again we're only working with text in this case, we need to verify the items are the correct type (in this case, UTType.plaintext).

import UniformTypeIdentifiers
import SwiftUI

class ShareViewController: UIViewController {
    override func viewDidLoad() {
       ...
        
       let textDataType = UTType.plainText.identifier
       if itemProvider.hasItemConformingToTypeIdentifier(textDataType) {
       // Load the item from itemProvider
       itemProvider.loadItem(forTypeIdentifier: textDataType , options: nil) { (providedText, error) in
            if error != nil {
                self.close()
                return
            }
               if let text = providedText as? String {
                 // this is where we load our view
               } else {
                   self.close()
                   return
               }
        }
}

Next, let's define our view! Create a new file, ShareViewExtension.swift. We are just editing text in here, so it's pretty darn simple. We just need to make sure we add a close() function that calls NotificationCenter so we can close our extension from the controller.

import SwiftUI

struct ShareExtensionView: View {
    @State private var text: String
    
    init(text: String) {
        self.text = text
    }
    
    var body: some View {
        NavigationStack{
            VStack(spacing: 20){
                Text("Text")
                TextField("Text", text: $text, axis: .vertical)
                    .lineLimit(3...6)
                    .textFieldStyle(.roundedBorder)
                
                Button {
                    // TODO: Something with the text
                    self.close()
                } label: {
                    Text("Post")
                        .frame(maxWidth: .infinity)
                }
                .buttonStyle(.borderedProminent)
                
                Spacer()
            }
            .padding()
            .navigationTitle("Share Extension")
            .toolbar {
                Button("Cancel") {
                    self.close()
                }
            }
        }
    }
    
   // so we can close the whole extension
    func close() {
        NotificationCenter.default.post(name: NSNotification.Name("close"), object: nil)
    }
}

Back in our view controller, we import our SwiftUI view.

import UniformTypeIdentifiers
import SwiftUI

class ShareViewController: UIViewController {
    override func viewDidLoad() {
       ...
               if let text = providedText as? String {
                   DispatchQueue.main.async {
                       // host the SwiftUI view
                       let contentView = UIHostingController(rootView: ShareExtensionView(text: text))
                       self.addChild(contentView)
                       self.view.addSubview(contentView.view)
                       
                       // set up constraints
                       contentView.view.translatesAutoresizingMaskIntoConstraints = false
                       contentView.view.topAnchor.constraint(equalTo: self.view.topAnchor).isActive = true
                       contentView.view.bottomAnchor.constraint (equalTo: self.view.bottomAnchor).isActive = true
                       contentView.view.leftAnchor.constraint(equalTo: self.view.leftAnchor).isActive = true
                       contentView.view.rightAnchor.constraint (equalTo: self.view.rightAnchor).isActive = true
                   }
               } else {
                   self.close()
                   return
               }
     }
}

In that same function, we'll also add an observer to listen for that close event, and call our close function.

NotificationCenter.default.addObserver(forName: NSNotification.Name("close"), object: nil, queue: nil) { _ in
   DispatchQueue.main.async {
      self.close()
   }
}

The last thing you need to do is register that your extension can handle Text. In your info.plist, you'll want to add an NSExtensionAttributes dictionary with an NSExtensionActivtionSupportsText boolean set to true.

You should be able to use this code as a foundation to accept different inputs and do different things. It's a jumping-off point! Hope it helps.

I had an old TV lying around, so I mounted it on my wall vertically. I grew up on StatusBoard, which was especially invaluable in newsrooms in the early aughts (gotta make that number go up!). I figured as I got deeper into self-hosting and my homelab I'd want some sort of status board so I could visualize what all was running, and partially just because everybody gets a dopamine hit from blinkenlights when they buy new stuff.

I was wrong! I in fact don't care what services are running or their status - I'll find that out when I go to use them. And since I mounted it on the wall, it wasn't particularly helpful for actually connecting to the various computers for troubleshooting. So I had to find something to do with it.

I loaded Dakboard on it for a while, which is pretty nice for digital signage. If I actually wanted to show my calendar, I would have stuck with them to avoid having to write that integration myself. But since my calendar already lives on my watch, in my pocket and in my menubar, I decided I didn't need it on the wall as well. And who wants to spend $4 on a digital picture frame???

So I built my own little app. I spun up a plain Typescript project, wrote an RSS parser, connected a few free photo APIs (and scraped the Apple TV moving wallpapers), and connected to my Plex server through Tautulli to get the data about what was currently playing. I got all of it wired up and ...

I hated it. Too much whitespace visible, and I felt compelled jack up the information density to fill the space. Otherwise, it was just sitting there, doing nothing. I for a second half-wished I could just throw up an old iPhone on the wall and be done with it.

And that's when it struck me. Why not use some physical design traits? Though skeumorphism got taken too far after the iPhone was first released, it feels we overcorrected somewhat. There's something to be said for having a variety of metaphors and interfaces and display options.

So that's where my first draft took me.

Honestly, I really like it! I like the aesthetics of the older iPod, seeing actual layers of things and some visual interest where the metaphor holds together visually. It's giving me serious "faux VR vibes" nostalgia like the software from the early 00s such as Win3D.

But I couldn't stop there. After all, I'd get burn-in if I left the same images on the screen for too long. So, every 12 minutes or so, when the image/video updates, there's a 50% chance the screen will shift to show the other view.

No vendor lock-in, here!

Note: This site now runs on Statamic

I knew I needed a new website. My go-to content management system was no longer an option, and I investigated some of the most popular alternatives. The first thing to do, as with any project, was ascertain the requirements. My biggest concerns were a) ability to create posts and pages, b) image management, and c) easy to use as a writer and a developer (using my definition of easy to use, since it was my site).

I strongly considered using Drupal, since that's what we (were, until a month ago) going to use at work, but it seemed like a lot of work and overhead to get the system to do what I wanted it to. I (briefly) looked at Joomla, but it too seemed bloated with a fairly unappealing UI/UX on the backend. I was hopeful about some of the Laravel CMSes, but they too seemed to have a bloated foundation for my needs.

I also really dug into the idea of flat-file CMSes, since most (all) of my content is static, but I legitimately couldn't find one that didn't require a NodeJS server. I don't mind Node when it's needed, but I already have a scripting language (PHP) that I was using, and didn't feel like going through the hassle of getting a Node instance going as well.

(Later on I found KirbyCMS, which is probably what I'm going to try for my next client or work project, but I both found it too late in the process and frankly didn't want to lose out on the satisfaction of getting it running when I was ~80% of the way done.)

As I was evaluating the options, in addition to the dealbreakers, I kept finding small annoyances. The backend interface was confusing, or required too many clicks to get from place to place; the speed to first paint was insane; just the time waiting for the content editor to load after I clicked it seemed interminable. At the same time, I was also going through a similarly frustrating experience with cloud music managers, each with a vital missing feature or that implemented a feature in a wonky way.

Then I had an epiphany: Why not just build my own?

I know, I know. It's a tired developer cliche that anything Not Built Here is Wrong. But as I thought it over more, the concept intrigued me. I wasn't setting out to replace WordPress or Drupal or one of the heavy-hitters; I just wanted a base to build from that would allow me to create posts, pages, and maybe some custom ideas later down the road (links with commentary; books from various sources, with reviews/ratings). I would be able to keep it slim, as I didn't have to design for hundreds of use cases. Plus, it would be an excellent learning opportunity, that would allow me to delve deeply into how other systems work and how I might improve upon them (for my specific use case; I make no claim I can do it better than anyone else).

Besides, how long could it take?

Four months later, LinkCMS is powering this website. It's fast and light, it can handle image uploads, it can create pages and posts ... mostly. Hey, it fulfills all the requirements!

Don't get me wrong, it's still VERY MUCH a beta product. I am deep in the dogfooding process right now (especially with some of the text editing, which I'll get into below), but I cannot describe the satisfaction of being able to type in the URL and see the front end, or log in to the backend and make changes, and know that I built it from the ground-up.

LinkCMS is named after its mascot (and, she claims, lead developer), Admiral Link Pengin, who is the best web developer (and admiral) on our Technical Penguins team.

I don't want to go through the whole process in excruciating detail, both because that'd be boring and because I don't remember everything with that many details anyway. I do, however, want to hit the highlights.

• Flight is a fantastic PHP routing framework. I've used it for small projects in the past, and it was pretty much a no-brainer when I decided I wanted to keep things light and simple. It can get as complicated as you want, but if you browse through the codebase you'll see that it's fairly basic, both for ease of understand and because it was easier to delete and edit routes as separate items.

• LinkCMS uses the Twig templating system, mostly because I like the syntax.

• The above two dependencies are a good example of a core principle I tried to keep to: Only use libraries I actually need, and don't use a larger library when a smaller one will do. I could have thrown together a whole CMS in Laravel pretty quickly, or used React or Vue for the front end, but it would have come at the expense of stability and speed, as well as (for the latter two) a laborious build process.

• I don't hate everything about WordPress! I think block-based editing is a great idea, so this site is built on (custom) blocks. My aim is to have the content be self-contained in a single database row, built around actual HTML if you want to pull it out.

• One of my favorite features is a Draft Content model. With most CMSes, once a page is published, if you make any changes and save them, those changes are immediately displayed on the published page. At best, you can make the whole post not published and check it without displaying the changes to the public. LinkCMS natively holds two copies of the content for all posts and pages - Draft and Published. If you publish a page, then make edits and save it, those changes are saved to the Draft content without touching the Published part. Logged-in users can preview the Draft content as it will look on the page. Once it's ready, you can Publish the page (these are separate buttons, as seen in the screenshots) for public consumption. Think of it as an integrated staging environment. On the roadmap is a "revert" function so you can go back to the published version if you muck things up too much.

• One of the things that was super important to me was that everything meet WCAG AA accessibility. Making this a goal significantly limited my options when it came to text editors. There are a few out there that are accessible, but they are a) huge (like, nearly half a megabyte of more, gzipped) and b) much more difficult to extend in the ways I wanted to. Again, with a combination of optimism (I can learn a lot by doing this!) and chutzpah (this is possible!), I decided to write my own editor, Hat (named after Link's penguin, Hat, who wears the same hat as the logo).
  I'm really pleased with how the Hat editor turned out, though it does still have some issues I discovered while building this site that are in desperate need of fixing (including if you select text and bold it, then immediately try to un-bold it, it just bolds the whole paragraph). But I'm extremely proud to say it that both HatJS and LinkCMS are 100% WCAG AA 2.1 accessible, to the best of my knowledge.

• Since I was spending so much time on it, I wanted to make sure I could use LinkCMS for future projects while still maintaining the ability to update the core without a lot of complicated git-ing or submodules. I structured the project so that core functionality lives in the primary repo, and everything else (including pages and posts) live in self-contained Modules (let's be real, they're plugins, but it's my playground, so I get to name the imaginary territory). This means you can both update core and modules, AND you only need to have those components included that you're actually using.

• I used a modified Model-View-Controller architecture: I call the pieces Models, Controllers and Actors. Models and Controllers do what you'd expect. Actors are what actually make changes and make things work. It's easier for me to conceptualize each piece rather than using "View" as the name, which to my mind leaves a lot of things out. I'm aware of the MVAC approach, and I suppose technically the templates are the View, but I lumped the routes and templating in under Actors (Route and Display, accordingly), and it works for me.

I don't think LinkCMS is in a state where someone else could install it right now. (For starters, I'm fairly certain I haven't included the basic SQL yet.) The code is out there and available, and hopefully soon I can get it to a presentable state.

But the end goal of all this was, again, not to be a CMS maven challenging the incumbents. I wanted to learn more about how these systems work (the amount of insight I gained into Laravel through building my own is astounding, to me), and craft a tool that allows me to build small sites and projects, on my own terms, with minimal dependencies and maximum stability.

Mission accomplished.

Email newsletters are the future. And the present. And also, at various points, the past. They've exploded in popularity (much like podcasts), hoping that individual creators can find enough people to subscribe to keep them afloat (much like podcasts). It's an idea that can certainly work, though I doubt whether all of the newsletters out there today are going to survive, say, next year, much less in the next 5. (Much like ... well, you get it.) My inbox got to the point where I could find literally dozens of new issues on Sunday, and several more during each day of the week. They were unmanageable on their own, and they were crowding out my legitimate email.

In a perfect world, I could just subscribe to them in Feedly. I am an unabashed RSS reader, with somewhere in the vicinity of 140 active feeds. I am such a hardcore RSS addict that I subscribed to Feedly Pro lifetime somewhere in the vicinity of ... 2013, I think. Gods. It was a great deal ($99), but it means that I miss out on some of the new features, including the ability to subscribe to newsletters. There are also some services out there that seem like they do a relatively good job, but even at $5/month, that's $5 I'm not sending to a writer.

And frankly, I was pretty sure I could build it myself.

Thus was born Newslurp. It's not pretty. I will 100% admit that. The admin interface can be charitably described as "synthwave brutalist." That's because you really shouldn't spend any time there. The whole point is to set it up once and never have to touch the thing again. The interface really only exists so that you can check to see if a specific newsletter was processed.

It's not perfect. There are some newsletters that depend on a weirdly large amount of formatting, and more that have weird assumptions about background color. I've tried to fix those as I saw them, but there are a lot more mistakes out there than I could ever fix. Hopefully they include a "view in browser" link.

Setup is pretty easy.

• Install dependencies using Composer

• Use the SQL file in install.sql to create your database

• Set up your Google API OAuth 2 Authenticaton. Download the client secret JSON file, rename it "client_secret.json" and put it in the project root

• Navigate to your URL and authenticate using your credentials

• Set up a filter in your Gmail account to label the emails you want to catch as "Newsletters." You can archive them, but do not delete them (the program will trash them after processing)

• Visit /update once to get it started, then set up a cron to hit that URL/page however frequently you'd like

That's ... that's pretty much it, actually. It worked like a charm till I started using Hey (which has its own system for dealing with newsletters, which I also like). But it still runs for those of you out there in Google-land. Go forth and free your newsletters!

Check out the repo here.

I have used WordPress for well over a decade now, for both personal and professional projects. WordPress was how I learned to be a programmer, starting with small modifications to themes and progressing to writing my own from scratch. The CMS seemed to find a delicate balance between being easy to use for those who weren't particularly technically proficient (allowing for plugins that could add nearly anything imaginable), while also allowing the more developer-minded to get in and mess with whatever they wanted.

I would go as far as to call myself a proselytizer, for a time. I fought strenuously to use it at work, constantly having to overcome the "but it's open-source and therefore insecure!" argument that every enterprise IT person has tried for the past two decades. But I fought for it because a) I knew it, so I could get things done more quickly using it, and b) it did everything we wanted it to at no cost. Who could argue against that?

The problems first started around the WordPress API. Despite an upswell of support among developers, there was active pushback by Matt Mullenweg, in particular, about including it in Core and making it more widely available - especially confusing since it wouldn't affect any users except those that wanted to use it.

We got past it (and got the API into core, where it has been [ab]used by Automattic), but it left a sour taste in my mouth. WordPress development was supposed to be community-driven, and indeed though it likely would not exist in its current state without Automattic's help, neither would Automattic have been able to do it all on its own. But the community was shut out of the decision-making process, a feeling we would get increasingly familiar with. Completely blowing the up the text editor in favor Gutenberg, ignoring accessibility concerns until an outside third-party paid for a review ... these are not actions of product that is being inculcated by its community. It's indicative of a decision-making process that has a specific strategy behind it (chasing new users at the expense of existing users and developers).

Gutenberg marked the beginning of the end for me, but I felt the final break somewhere in the 5.x.x release cycle when I had to fix yet another breaking change that was adding a new feature that I absolutely did not need or want. I realized I was not only installing plugins were actively trying to keep changes at bay, I was now spending additional development time just to make sure that existing features kept working. It crystallized my biggest problem I'd been feeling: WordPress is no longer a stable platform. I don't need new; I can build new. I need things to keep working once they're built. WordPress no longer provides that.

And that's fine! I am not making the argument that Automattic should do anything other than pursue their product strategy. I am not, however, in their target market, so I'm going to stop trying to force it.

Note: This was the write up a of a conference talk given in 2016, and should be considered of its time/not used for current dev practices or advice. Don't use WordPress.

The WordPress REST API is the future! This is something many of us have been saying/believed for about two years, but the future is now! Kind of. The REST API has (finally) been approved for merge into WordPress 4.7, meaning it will be available for use by everyone without requiring a plugin, as has been the case up to this point. Even without the official recognition (and with a not-small barrier to entry), lots of people and companies have done some pretty amazing things with the REST API. So I thought we'd look into the things people have done to get ideas for what the API will be useful for, and other ideas that might be best solved other ways.

For the purposes of this discussion, WordPress can be defined by an easy-to-use, extensible content management system. It powers a staggering amount of the internet, customized in an equally dizzying array of ways, and runs on PHP. The PHP part is often the biggest objection people have, for many reasons (not available in their stack, they don't like the language, etc.).

The two people who are co-leading development on the REST API (Rachel Baker and Ryan McCue) wanted to "future-proof" WordPress by allowing the development of new features and enhancements as well as people who were outside the PHP ecosystem to use WordPress.

That's a longer way of saying, "The REST API was invented so everyone doesn't have to rely on the traditional theming/plugin structure of Wordpress." The infrastructure for the REST API was implemented in WordPress 4.4. The first set of content endpoints (making the API actually useful to anyone) have been officially approved for integration into 4.7. For those still following along, that means that, with the release of 4.7, the following endpoints will be available for read/write:

• Posts

• Comments

• Terms

• Users

• Meta

• Settings (through Options)

Obviously, some of the specific data (such as User information and Settings) are restricted by user. The publicly available API will not return information that is otherwise locked-down in WordPress. Don't forget to install some sort of authentication plugin (Oauth1.0, Oauth2.0, JWT, application passwords, whatever floats your boat), as none made it into the 4.7 update (though they're aiming for 4.8).

In general, the API is going to be godsend whenever you're trying to work with WordPress and you want to break out of the traditional PHP/theme/plugin interface. In good news for the diehard PHP developers out there, you don't have to change a darn thing if you don't want to!

Specifically, I think the best thing going for WordPress right now is its ubiquity and ease of use in creating content. A ton of people have interacted with the WordPress editor at this point, and it's pretty intuitive to anyone who's ever opened Word. It's a fantastic tool to allow people to enter content. And plugins like Advanced Custom Fields or TablePress make filling out what otherwise might be tedious and confusing data (say, product details) remarkably simple. It's then up to us as developers to take that content and make it compelling to whoever's going to be reading it.

Previously, we've been restricted to using the traditional WordPress theme/plugin model — which is not a knock against it! People have built absolutely astounding things using it. But in order to do anything else required XMLRPC or writing your own custom endpoints, for which there was no standard and you'd have to manually include every piece you want.

Probably the biggest change that's going to hit is the ability to craft themes with zero use of PHP. Are you a React developer? Angular? Now, as long as the style.css is set up and loading in the JS files, you can create an entire theme without every having to write a single the_content();. A corollary to this idea is ...

The best part is, since WordPress is no longer chaining your front- and back-end together, it's extremely easy to use WordPress as just one part of your stack and use it only for specific needs it's well-suited for.

I work at the Penn State College of Medicine on our Hershey campus, which encompasses both the college part as well as the Milton S. Hershey Medical Center. The Marketing Department is responsible for both entities. As you can imagine, our tendrils reach all over the place: We run the public web site, the intranet, TV screens all over the place running a bastardization of PowerPoint (the content is a combination of calendar display, events, announcements and tons of other information), print flyers, print brochures, email marketing, signage, news, public events ...

Obviously, we're not going to be able to incorporate ALL of that into one place, and rightfully so — there are legitimate reasons InDesign exists, and we don't need to pull in print production to this process. But the ability to pull our digital together where that makes sense? I'm swooning. WordPress, making its content available through the API, could absolutely be the nervous system that gets all of our content going out to the various appendages from the central brain.

Enter the event in the backend, tag it appropriately, add the image. Now it's on the events calendar website immediately for people to find. Three weeks before it happens, we send out an automated email (using our CRM, not publishing out via MailPoet), push to the intranet and it goes up on the screens so everyone walking around can see it. And the information we send out is standardized and coming from the same location every time. We can integrate it wherever else it makes sense, and let the other systems are already in place work around/with the data.

And that's just my specific use case. Ryan McCue suggested touch-screen museum displays, but the more obvious implementation comes from ...

I'm sure at some point between 2009-2013, we all encountered the "Download our app!" website on a WordPress site that was little more than a basic blog. Someone (many someones, actually) decided that every website should have their own native app.

The easiest way (short of legitimately scraping the content from a website, which is time-consuming/hard) of doing this involved picking up one or more RSS feeds from the site in question and displaying it in a questionably-slightly-easier-to-use format. I worked for a publisher who went through more than the three companies mentioned by link earlier on in this paragraph, and I can tell you there were many aspects that drove us absolutely nuts, almost all related to customization.

We were allowed one (!!) image per article, because that's the standard featured image RSS feeds pump out. We weren't allowed to use JavaScript because they didn't know/didn't care about how to accurately parse the CDATA, and anyway they didn't have the right libraries loaded (or weren't willing to let us load libraries on the page). What we wouldn't have given for the ability to ship them a standard set of JSON, along with whatever custom parameters/images/etc. we wanted, and told them to design for that.

Luckily, even though this is about to get much easier for the average user, I think we as a society will be able to dodge the "have a WordPress site? Get a native iOS app up and running in 10 minutes!!!!" hucksters, simply because most people have already figured out that there's no real advantage (financial or otherwise) to having an app unless you have a reason to have an app. (Also, someone's already doing this.)

But imagine the people who DO need an app. Suddenly you can pull all your product information out of WooCommerce with ease and set it up to support native purchases using the Apple App Store or Google Play Store right on someone's phone (or the Windows Store and Mac Store, I guess, if you're into that sort of thing).

While most of us rightly blanch at the "advantage" of giving someone 30% of revenue, it's simple fact that for a lot of people that defines online shopping. That specific distribution model isn't going to work for anyone, but the ability to liberate information from the core install (and update it back in when something changes) opens up the possibilities for native applications the same way it does for non-PHP languages on the web.

While I will sing the praises of the content editor all day long, I give a deep shudder of foreboding when contemplating what permissions levels to set up for client users. Naturally, they all want to be admins ("It's my site, after all"), and all I can think about is how quickly they'll try to delete plugins, or change the theme ("it's cooler!"), or any number of problems. That doesn't even get to the non-project owners, who still want admin access, or who get lost because "I couldn't find that post!" (because it's a page).

With the endpoints we have now, we will be able to more easily only surface to people exactly what they need to be monkeying with. A content editor can get in and see the content editing screen, and no more. Even better, you can create detailed experiences for users at scale where they can manage their account information without having to drop them into the WordPress backend. Or, you can integrate some of the WordPress account information into wherever the user already is (say, in your ASPX-powered webapp's user screen). The key, as is the recurring theme here, is external extensibility. No longer are we confined to the WordPress sandbox. Speaking of which ...

I'm unapologetic in my love for the WordPress editor, but I recognize that it's not a tool that's going to work for everyone in every situation. Some people really need to see how the content is going to look before they feel comfortable with it, and now they can.

Front-end editing just got a LOT easier. Scrolling through your site and see a typo? If you're logged in, all you need to do is the "edit this page" link, make your edits, save the page, and continue scrolling on. Similarly, this makes applications similar to Calypso available without having to run JetPack. Now you can customize a WordPress iOS app to manage your posts and your custom fields, without having to worry about marrying everything up as you try to sync back.

Many stores (think Sears, or Verizon) now have their employees wandering around with tablets for various reasons: mobile checkout, customer service triage, etc. Let's take customer service triage: one WordPress install, an API-powered screen showing who's registered/approximate waiting time/whatever, backed by a native (or web-powered, doesn't matter) app running on the representatives' tablets that allows them to register and manage the queue.

The exciting part of this is absolutely not trying to get Verizon to replace their system (honestly don't care), but rather the ability to bring it to your grocery store for their meat counter, or maybe your local independent bookseller who wants to showcase different things on screens around her store. It's making the implementation easier on a smaller scale, cheaper, and growing the ability for people who already have an existing skillset to take advantage of it.

The second generation of the web (web 2.0, whatever you want to call it) was built on sharing data and user generated content. There have always been ways to integrate this into WordPress, but a full extensible API blows that out of the water. When I worked at a newspaper, user-submitted photo galleries did huge traffic for us, but we were using a platform that wasn't really designed for photo galleries (and we were going to stop paying for it).

At the time, I built a wrapper around the SmugMug API, but if I had an API for WordPress available at the time I probably would have used that instead. Imagine a drag-and-drop box where you put your photos in, tell us the captions, and they're automatically uploaded to the backend and ready for us to examine and approve them. All of the authentication, administration and admin work would have already been handled out of the box. It's not necessarily creating a whole new paradigm for the internet, but it does extend WordPress' capabilities to meet existing needs, and make it easier on everyone.

Most of the implementations we've been talking about thus far are focused on getting information out of the WordPress install for various purposes — one good one that publishers might want to focus on is AMP, Facebook Instant and Apple News pages. Since you can now grab your data via the API, pushing the information required by those services just got a lot easier. But we're talking about a full REST client, here. Incoming signals can be gathered from whatever other services you're collecting from (Facebook, Twitter, FourSquare, whatever), pushed back into the WordPress database for storage (and then extracted back to whatever you want using the same API).

Supporting WordPress just got a whole lot easier. Doing basic work/upgrades on a bulk scale have gotten easier and easier with automatic updating and WP-CLI, the REST API (if extended to the full use that its founders envision) could accelerate those changes by orders of magnitude. Enter RESTful WP-CLI. This project (which is in very early stages and, as its README warns, "breaking changes will be made without warning. The sky may also fall on your head.") is the type of innovation supported by the API that will save all us tons of time in the future. It automatically extends the REST endpoints into WP-CLI, allowing you to make the same changes from the command line.

The best part is, these tools will allow us to gain benefits even without expensive retrofits of existing sites. I'm certain there exist now organizations that run more than one WordPress installation that's not on multisite (for whatever reason), and don't have the time/money/internal political clout to change that. Simply through core updates to WordPress and the installation of a single plugin (RESTful WP-CLI), you could write a batch script to SSH into your WordPress host(s) and automatically add or delete a user the next time your company makes an HR change, and only have to do it once.

Right off the bat, the REST API will give us numerous ways to automate certain processes, and that number will only grow as the API gradually extends/eats the rest of WordPress' functionality.

It's not all upside, of course. Due to inherent limitations in the current implementation (lack of access to menus, single authentication option), not everything is going to feasible or wise in terms of build-out.

As of when the REST API is integrated into the core, there's not full wp-admin parity baked in. If your project operates outside the traditional theme model, basic structures and actions we've taken for granted such as the permalink and post previews available to users via the backend will no longer work unless the developer goes out of their way to reimplement the basic functionality.

Similarly, menu management is not going to be baked-in, at least in 4.7, which means you're going to need to do a little work (API Menu plugin) or a lot of work (recreating it however works best for you by hand) in order to get those things working. If you're just feeding information into somewhere else that's managing those things, no sweat. But if you're running that information into an iOS app, for example, you're going to need to deal with in one way or another, and the way people are used to (Appearances > Menus) isn't going to work.

That being said, the biggest caution flag I can see for developers is wanting to use the new API where it isn't necessarily needed. As mentioned several years ago in the TL;DR, maybe the project you're working on is specifically so you can see how the REST API works. If so, great, go nuts.

If, however, you're just trying to get the job done and you need to surface related posts inside a regularly-built theme, make sure you actually need to be calling those in via the API versus just including a custom loop in the footer, or something similar. I don't know if most of you know this, but developers of all stripes have a tendency to jump on something and use it for their next project because it's new, or someone said it's faster, or (often) just because it exists. The API enables us to do a lot of cool new things, but it doesn't necessarily need to replace your entire existing workflow. Use the tools best for the job at hand, and plug in new ones as necessary.

While this whole post has been API cheerleading and the ability to stop relying on PHP, I nonetheless want to caution people to think before implementing everything on the client side. There are many useful additions, hacks and modifications you can make on the server side to speed up the client side. Filters, long a valuable asset in the theme/plugin builder's toolbox, are even more so when it comes to the API. The large number of hooks available give you options, and it's up to you exploit them.

A project I did had a number of Advanced Custom Fields attached to a custom post type, including an image (with caption) and a relationship field to another post. With the proper hooks, I was able to transform the single call for an "event post" type to include the image, caption, the full content of the relationship post, and all of its ACFs. One call, all the info I needed (and I did need all the info every time I called it). A roundabout way of saying that just because PHP isn't necessarily going to be what you use to build the majority of the project, it doesn't mean you should discard it entirely. Your current WordPress skills aren't going by wayside, you're just finding new ways to augment them.

Examples mentioned

• Quick Draft

• Press this

• Times of London

• Wired

Resources

• Github Repo - The repository for the questions theme from php[world]

• Plugins - WordPress plugins to extend the REST API

• Ionic — a mobile UI framework (Angular-based)

• Vienna — iOS native editor for WordPress (React Native)

• WP REST API React/Redux helpers — data-management helpers for React and Redux as Node modules

• node-wpapi — WordPress REST API client for JavaScript

It’s probably the most standard Twitter profile text outside of ostensibly nubile 22-year-olds who are “just looking for a guy to treat me right” — “Retweets are not endorsements.” Journalists, who are among the more active Twitternauts, like to pretend they exist outside of normal human functioning like judgment and subjectivity, and thus use this phrase to let everyone know that just because they put something on their personal (or corporate-personal) account, it doesn’t mean THEY actually think that thing. They’re just letting you know. It’s FYI.

It’s bullshit.

This is the ignore-the-obvious-fiscal-advantage argument that’s given whenever people wonder why the media focuses on inane, unimportant or crazy stories that even most journalists are sick of — sometimes even on air. We know that you posted the story about the celebrity because people will click on the link about the celebrity. It’s why the concept of clickbait headlines exist: it’s certainly not for the reader’s benefit. Journalists have ready-made reasons (read: excuses) as to why they post tripe, and the closest they ever get to the truth is “because people will read them.” . They’re just trying to inform people!

With the democratization of communication accelerated by the internet, “major media” no longer holds any meaningful gate-keeping role in deciding what people should know about. You can lament or celebrate this information as you may, but most would not argue with the truth of it. There are simply too many outlets through which you can acquire information, be it personal feeds from social media, websites, TV channels, magazines, etc. If someone wants to get their message out into the world, there are ample ways to do this.

Let’s take, for example, an American neo-Nazi group. Their message is that the white race is superior and other races should be subjugated/deported/killed. They might have a Twitter account, a website, a magazine, whatever. The main point is, none of these mediums have the ability to reach out to people. Sure, they can tweet @ someone and force their way in, but for the most part the way people interact with their message is through (digital or actual) word-of-mouth from those who espouse those beliefs, or by seeking them out directly.

But what happens when, say, a major party presidential candidate retweets some of their views? It by no means indicates that the candidate himself is a white supremacist or in any way sympathetic to those points of view. But it does give the jerks a voice. It lets people who may similarly not be white supremacists or sympathizers be exposed to that person, and provides them a vector to that information. Clicking on the Twitter handle to see the white supremacist’s past tweets opens the door. The person who goes through it is not automatically going to become a skinhead … but perhaps that Twitter user is adept at using misleading rhetoric and subtle innuendo to draw people down the path.

None of this makes it the candidate’s fault (or the candidate a racist [UPDATE: Except when it does, don't slow-walk that nonsense, Past Me]), but the root cause is undeniable.

So what does this have to do with the media? The sole ability any publication/outlet has is to determine what information they think their readers should know. They cannot make their readers know this information anymore than the presidential candidate or the racist twit can make anyone pay attention to them. All they can do is put the information in front of those who let them. It’s exactly the application of, “You can’t control what other people think, you can only control what you do,” only this time it has nothing to do with telling your child that some people are just mean.

The story is whatever the story is, and by printing a story in the newspaper, airing it on your broadcast network or pushing it to your audience via Facebook, your website, YouTube, etc., the publisher/creator is saying “This is a thing that is worthy of attention.” Especially if you’re not going to put any effort into context (which is what a retweet is), you’re explicitly stating to your audience that this is a thing they should know about. In an “attention economy,” with a surfeit of content and not enough eyeballs, getting someone to look at you goes a big way toward your winning (whatever it is you’re trying to win).

Thus, tweets like this:

Newsrooms insisting, "No, re-tweets ARE endorsements" have really said: we don't trust our journalists or our users. http://t.co/gX923Ej9rN

— Jay Rosen (@jayrosen_nyu) July 11, 2014

are actively missing the point. No one’s saying you absolutely believe 100% in whatever you retweet. But it’s disingenuous to argue that there’s no value to the original tweet by your retweet. Hell, if there wasn’t, there would be no point in your retweeting it at all.

As a person whose life is consumed by the digital world, this feels an exceptionally strange piece to write. I spend the vast majority of my day on a device, whether that’s a computer for work (I’m a web developer, no escaping it) or a phone/computer/tablet for whatever (likely cat-related) thing I happen to be internetting in my free time.

So you can understand my internal consternation when confronted with a situation that makes me lean toward limiting technology. I’m more than a little worried about technology, both for the reaction it’s drawing as well as its actual impact it’s having on society as a whole — and not just because three out of every four stories on every single news site is about Pokemon Go.

But we’ll get there. First, let’s start with something more mainstream.

Technology (and, more specifically, apps/the internet) are famous for disruption. Tesla’s disrupting the auto industry. So’s Uber. AirBnB “disrupted” the hotel industry by allowing people to rent out rooms (or entire houses) to perfect strangers. The disruption in question (for hotels) was that they no longer were the combination of easiest/cheapest way to stay in a place away from home. But there was also “disruption” in terms of laws/regulation, a fight AirBnB is currently waging in several different locations.

Some of these fights revolve around leases — many landlords do not allow subleasing, which is what some people do on AirBnB: Rent out a space they rent from someone else for a period of time. AirBnB asks that people confirm they have the legal right to rent out the space they’re listing, but there’s no enforcement or verification of any kind on AirBnB’s part. AirBnB thus, at least in some non-small number of cases, is profiting off of at best a breach of contract, if not outright illegality. Then there’s the fact that anyone, be they murderer, sex offender or what have you, can rent out their space and the person renting the room may be none the wiser.

And maybe these things are OK! Maybe it should be caveat emptor, and the people who ultimately lose out (the actual lessees) are the ones primarily being harmed. But that ignores the people who were just trying to rent from AirBnB and had to deal with an irate landowner, or the property owner who has to deal with the fallout/repercussions of the person breaking the lease.

The clichéd technical model of “move fast and break things” should have some limits, and situations where people are dying need more foresight than “we’ll figure it out as we go along.” Otherwise, how do we determine the appropriate death toll for a new tech service before it needs to ask permission rather than forgiveness? And before you dismiss that question as overbearing/hysterical, remember that actual human beings have already died.

But not everything is so doom and gloom! Why, Pokemon Go is bringing nerds outside, causing people to congregate and interact with one another. It’s legitimately fun! Finally my inner 10-year-old can traipse around the park looking for wild Pikachu to capture. Using augmented reality, the game takes your physical location and overlays the game on top of it. As you walk around with your phone, it uses your GPS location to pop up various Pokemon for you to capture. There are also Pokestops, which are preset locations that provide you with in-game items, located in numerous places (usually around monuments and “places of cultural interest”). There are also gyms in similarly “random” places where you can battle your Pokemon to control the gym.

And no deaths! (Yet, probably.) But just because no one is dying doesn’t mean there aren’t still problems. Taste-wise, what about the Pokestop at Ground Zero (or this list of weird stops)? Business-wise, what about the Pokestop near my house that’s in a funeral home parking lot? You legally can’t go there after-hours … but Pokemon Go itself says that some Pokemon only come out at night. What happens during a funeral? There’s no place where businesses can go to ask to be removed as a Pokestop (and frankly, I can imagine places like comic book stores and such that would pay for the privilege). And who has the right to ask that the 9/11 Memorial Pool be removed? Victims’ families? There’s an appropriation of physical space going on that’s not being addressed with the seriousness it should. Just because in the Pokemon game world you can catch Pokemon anywhere doesn’t mean, for example, that you should necessarily allow have them popping up at the Holocaust Museum.

I would like to preempt arguments about “it’s just an algorithm” or “we crowd-sourced” the information by pointing out that those things are useful in their way, but they are not excuses nor are they reasons. If you decide to crowd-source information, you’d better make sure that the information you’re looking for has the right level of impact (such as the names of boats, or in Pokemon Go’s case, the locations of Pokestops). Some of these things can be fixed after the fact, some of them require you to put systems in place to prevent problems from ever occurring.

In this case, you can cast blame on the players for not respecting the law/common sense/decency, and while you’d be right, it shifts the blame away from the companies that are making money off this. What inherent right do companies have to induce people to trespass? Going further, for some reason doing something on “the internet” suddenly cedes rights completely unthinkable in any other context. Remember the “Yelp for people” that was all but an app designed to encourage libel, or the geo-mapping firm that set the default location for any IP address in the US to some Kansan’s front yard. These were not malicious, or even intentional acts. But they had very real affects on people that took far too long to solve, all because the companies in question didn’t bother (or didn’t care) about the real effects of their decisions.

At some point, there’s at the very least a moral — and should be legal, though I’m not necessarily advocating for strict liability — compulsion to consider and fix problems before they happen, rather than waiting until it’s too late. The proper standard probably lies somewhere around where journalists have to consider libel — journalists have a responsibility to only report things they reasonably believe to be accurate. Deadlines and amount of work are not defenses, meaning that the truth must take priority over all. For places where the internet intersects with the real world (which is increasingly becoming “most internet things”), perhaps a similar standard that defers to the reasonably foreseeable potential negative impact should apply.

Technology is only going to grow ever-more entrenched in our lives, and as its function moves closer to an appendage and away from external utility, it’s incumbent upon actors (both governmental and corporate) to consider the very real effects of their products. It (here meaning “life,” “work” or any number of quasi-existential crises) has to be about more than just making money, or the newest thing.

We’re all pretty much in agreement that racism is bad, yes? Even most casual racists will usually accede this point, right before clarifying how their racism isn’t actually racism. Or something.

But what, then, to do with the people who say bigoted things (be they related to race, gender, or whatever)? The easiest path would be to simply ostracize them, mock them, or otherwise diminish their roles in society. And this gets done all the time! (Ask Twitter.) And sometimes those (publicly, at least) repent of their ways and pledge to do better in the future, and life goes on.

And sometimes those people are dead.

(Please note: The views represented in this piece are intended to apply only to those who have already died. For living authors/comedians/people of note, it's a whole different situation.)

Woodrow Wilson is in the news again, because his name adorns the Princeton School of Public Policy and International Affairs. Wilson — the 28th president, Nobel Peace Prize winner, and a president of Princeton University — was also unequivocally, unquestionably racist. Because of this fact, Princeton students have demanded that Wilson’s name be taken off all programs and buildings.

Again, the easy path is simply to take his name off the building. But how do you erase a president from history? For that matter, how do you justify removing the name of the man who dreamed up the League of Nations (the forerunner to the United Nations) off of a school of international affairs? A man that won what is considered the biggest prize in human history (the Nobel Peace Prize) because of his work in international affairs?

To wit: how do you separate the man from his work?

I just finished The Secret History of Wonder Woman, a book more accurately titled The Secret History of the Creator of Wonder Woman, that dovetails quite nicely with this debate. William Marston was a failed psychologist/moviemaker/entreprenuer/inventor who created Wonder Woman.

The early comics (authored by him, before his death in 1947, were chock-full of progressive feminist ideals: WW solved problems by herself (never waiting for Batman or Superman to save the day); She actively refused marriage to her boyfriend; Her female friend, Etta Candy, on several occasions helps WW subdue her male foes.

The feminist ideal manifested itself in more obvious ways, too: WW shows a young boy the important role of women in history, WW helped the namesake of her alter ego out of an abusive relationship, and the earlier comics even included an insert printing of “Wonder Women of History,” a four-page adventure chronicling the lives of women such as Florence Nightingale, Susan B. Anthony and Helen Keller. Sounds like a pretty cut-and-dried case of progressive values that deserve to be lauded.

Of course, I wouldn’t have included it as an example without a very large “but." Marston married his wife, Elizabeth Marston, in 1915. He had an on-again, off-again relationship with Marjorie W. Huntley that his wife knew about — and lived permanently (along with Elizabeth and, infrequently, Huntley) with Olive Byrne, whom he presented with golden bracelets as an “anniversary gift”. (The bracelets are the inspiration for WW’s, and are thought to have symbolized their private “marriage.") Byrne’s role in the triad was to raise the children — eventually, two of her own and two by Elizabeth.

There’s nothing inherently wrong or bad about their living arrangement, of course — peoples’ private lives are their own. But one is forced to at least ponder the impulses for creating WW by a man who publicly claimed — in 1942, no less — that women would rule the world after a literal battle of the sexes … as he was financially supported by one wife and had a second at home who was tasked with taking care of the children. It’s entirely possible that Byrne desired this life and had no problem with it. It’s also possible that it’s the only arrangement Elizabeth would agree to.

Then there are the many, many instances of bondage WW undergoes, undergirded by Marston’s belief that women were naturally more submissive than men. But it was OK, because men could learn submission from women, who would rule over them with their sexiness: The only hope for peace is to teach people who are full of pep and unbound force to enjoy being bound ... Only when the control of self by others is more pleasant than the unbound assertion of self in human relationships can we hope for a stable, peaceful human society.

So was Marston a feminist? Or was he a sex-craved submissive longing for a dom? In either case, how does that change Wonder Woman? The answer, of course, is that it doesn’t. Authorial intent is absolutely important for discovering the reasons why something is written and for discerning its influences, but ultimately the work itself is judged by the individual reader.

It absolutely can make a difference in how the work is read (in that an individual will bring their own prejudices and biases just as they do in every instance of human reason), but only as much as the reader wants it to. Cultures and mores change. The esteem historical figures are held in wax and wane when they’re looked at with eyes that have seen the impact of past ignorance.

Some, like Christopher Columbus, are doomed to be relegated to the bigot wing of history because their accomplishments (finding a continent the vikings discovered hundreds of years earlier) are overshadowed by the way they accomplished them (indiscriminate slaughter and enslavement of indigenous people). Others, such as Abraham Lincoln, get their mostly exemplary record (freed the slaves!) marred by simply being of a certain time period (“... I will say in addition to this that there is a physical difference between the white and black races which I believe will forever forbid the two races living together on terms of social and political equality.”) and adopting a progressive stance (for the time), but still not getting all the way there.

That’s a good thing.

Historical figures and events are never as black and white as they’re presented in history classes. Shades of gray exist everywhere, just as they do in your everyday life. We present them simplistically for a variety of reasons, but nobody’s perfect.

So what do we do with Wilson? It’s never wrong to have a debate, to illuminate the issues of the past and the present. As to whether the name gets removed ... meh? Honestly, if the students are the ones who have to use it and they care so much, why not change it?

Buildings will ultimately crumble, institutions ultimately fail and time marches inexorably along. The best we can do is respect the past while always remembering that the needs of the present outweigh those of the dead. Events happen, with real consequences that need to be considered. But, ultimately, people are rarely all good or all bad. They are, after all, people.

I like technology. I think this is fairly obvious. I like it personally because it removes a lot of friction points in my life (some in ways that other people appreciate as more convenient, some in ways that are convenient only to me). But the downside of technology is that businesses use it as a way of not paying people for things that actually often do require human judgment.

The proper way most systems should be set up for, say, a medical insurance claim is that you fill out everything electronically so the data is in the right place and then an actual human can make an actual human judgment on your case. In practice, however, you fill out the form and the information whisks away to be judged by a computer using a predetermined set of rules.

If you're very, very lucky, there might be a way for you to appeal the computer's ruling to a human being (regardless of outcome/reason) — but even then, that person's power is often limited to saying, "well, the computer said you don't pass."

The following story is by no means of any actual consequence, but does serve as a prime example of how to waste your money employing customer service people. I recently switched banks. When I was at the branch doing so, I asked out of curiosity if they allow custom debit cards (my girlfriend has a credit card that looks like a cassette tape, and is always getting compliments on it. I'm petty and jealous, so I want a cool card, too).

Finding out the answer is yes, I waited until my actual debit card came so I can see the pure eye-rending horror that is their color scheme before sitting down and trying to make my own. I wasn't really looking to lose a good portion of my day to this endeavor, so I used the Designer's Prerogative to begin.

I wanted something computer-y (see above, re: my opinion on technology), so I started with this (royalty-free) stock image. Their design requirements say the PeoplesBank logo has to be large and colored (dark red for Peoples, gray for Bank), so I swapped the colors on the image and flipped it so the faux-binary wouldn't be covered by the big VISA logo or hologram (see the image at the top of the post).

It's not a masterpiece, it's not like I slaved over it for hours. It's just a cool design that I thought would work well. Upload, and send!

Three hours later, I got an email: SORRY — your design wasn't approved!

We regret to inform you that the image you uploaded in our card creator service does not meet the guidelines established for this service, so it has not been accepted for processing. Please take a moment to review our image and upload guidelines at www.peoplesbanknet.com and then feel free to submit another image after doing so.

Huh. Well maybe I ran afoul of the design guidelines. Let's see, competitive marks/names, provocative material (I don't think so, but who knows?), branded products ... Nope. The only thing that it could possibly even run afoul of is "Phone numbers (e.g. 800 or 900 numbers) and URL addresses (e.g. www.xyz.com)", but since it's clearly not either of those things, I figured it would be OK.

So I called up PeoplesBank and explained the situation.

"Hi, I was wondering why my custom card design was rejected."

"Well, it should have said in the email why it was rejected."

"Yes, it says 'it does not meet the guidelines established for the service.' I've read the guidelines and there's nothing in there that would preclude this. It's just an abstract image with some binary code, and it's not even real binary, it's just random 1s and 0s."

"Please hold."

[5 minutes pass]

"OK, it says the copyrighted or trademarked material part is what it ran afoul of."

"It's just numbers and an abstract image. How could that be the problem?"

"That's what it says."

"OK, well, is there someone somewhere I can talk to who would be able to tell me what I need to alter in order to make it acceptable?"

"Please hold."

[10 minutes pass]

"OK, you said something about the numbers? Something about by Mary?"

"Yes, it's binary code. Well, it's not even really binary, it's pseudo-binary."

"Well, that's it."

"What's it? It's just random 1s and 0s. It's the equivalent of putting random letters in a row and saying they're words."

"Apparently it's copyrighted."

"... OK, well, is there someone who can tell me what I need to change? Because I doubt that, even if I changed the numbers around and submitted it, it would still go through. I just need to know why it's not going through so I can change it so it does go through."

"Oh, we'll need to research that. Is there a number I can call you back at?"

My best guess is that somehow this is getting tripped up as an allusion or reference to The Matrix by some content identifier program somewhere, which a) it's clearly not, b) The Matrix wasn't actually binary, and c) you can't copyright the idea of code on a screen. The computer identified as such, and since no one actually knows why it thought that, no one can tell me how to fix it.

And since it's such an important business case (not getting sued for copyright infringement, even though there's absolutely no way VISA is getting sued even if someone puts Mickey on their damn credit card), no one is actually empowered to overrule the computer.

What I'll probably end up doing is just trying another image (I was thinking maybe a motherboard) because at this point I've already spent more time than I actually care about the design of my debit card. It's just frustrating.

I sincerely hope I don't have to update this post.

Frustration is a natural part of doing ... well, anything, really. Especially when you're picking up something new, there's almost always a ramp-up period where you're really bad, followed by gradual progression. You know this. I know this.

It's kind of obvious to everyone who's ever played a sport, an instrument or tried anything even remotely skilled. There's room for natural talent to make things a little easier, of course, but even LeBron James went through a period (much earlier and much shorter than the rest of us) where basketball was something new he had to get good at.

There are various schools of thought on how to approach this: Some believe people should be allowed to develop at their own pace and just enjoy the activity; others believe that screaming things at children that would make drill sergeants blush is the best way to motivate and/or teach them. Personally, I think the right approach falls somewhere in the middle (though toward the non-crazy side), depending on age, experience and what the person in question wants.

**All of which is a long-winded way of saying that a not-insignificant number of people who play videogames online are absolutely terrifying human beings. **

When I get the chance lately, I've been picking up and playing Rocket League, a game best described as "soccer with cars that have rockets in them." From a gameplay perspective, there's a decent amount of strategy involved that combines soccer with basketball. The single-player AI is pretty easy to defeat, though it does allow for a nice ramp-up of abilities and skills. Then there's the online portion.

Before this month, there were just random matches you could join (from 1x1 up to 4x4) and play against other people. Some of those people are clearly wizards, because they fly around and use the angles to pass and score from places that I would have trouble even mapping out on paper.

In this initial period, the random matches I joined (which is to say I didn't join any guilds or teams, just random online play, so there's some bias there) were mostly fun, occasional blowouts (in both directions) that often involved no more chatter than the preset options ("Great pass," "Nice shot," "Thanks," "Sorry," etc.).

Then, with an update this month, Rocket League rolled out rankings. Now you can play "competitively" in a division (stratified tiers to ensure that people of like ability play against one another) and receive an overall score of your skill level. And boy do a lot of people seem to think it's somehow indicative of their worth as human beings.

I play where everyone starts, in the unranked division. You start with 50 points and win/lose between 6-10 points per game you play, depending on the team outcome (important note). I currently bounce around the mid-to-upper part of this unranked tier, which is probably pretty accurate (I'm OK, but have moments where I screw up).

For the first few games I played, it was interesting watching the different skill levels (from brand new or just-out-of-single-player to pretty skilled players) interact with one another fairly frictionlessly. There'd be some frustrating boneheaded moves that might cost you a match, but it generally appeared to just be accepted as part of playing on a randomized team. When I played yesterday, though, things seemed to be getting ugly.

The first two matches went fine — a win, a loss. Then I got a string where I was teamed up with what one can uncharitably describe as spoiled babies.

In unranked play, the first one happened when I came out too far forward on defense and let a goal go by. Unquestionably my fault, which is why I shot off a "Sorry" to my teammates. "Fuck don't miss the fucking ball," was what I got in response.

We had another goal scored on us during the vagaries of play, as happens, because the other team was better than us. That's when my teammate got mad. "God you're terrible. You must be doing this on purpose."

Which isn't bad, as internet rantings go. It just caught me off-guard. He proceeded to score relatively soon after that to tie things up, and I flashed a "Nice shot!" to him. "fuck off, [gamertag]."

Um, OK.

In the very next match, we scored a quick goal to go up 2-1 when someone from the other team asked if they had removed a feature (he used more obscenities than my paraphrase). He then proceeded to rant about the "shit physics implementation" and how "he totally had it 100% locked-in."

Of course, given that he was typing all this while the game was still going, his team wound up giving up a few more goals, but his point definitely got made.

After an uneventful game following that, the last one involved a (clearly) new player whiffing on defense, and three players from both teams proceeded to disparage the player with accusations of "trolling" — losing on purpose — to the point where he just literally stopped playing. His car just remained motionless on the field.

It's easy to sit back and wonder about why they take it so seriously — "it's just a game" — but that's a simplistic answer. I have no problem with taking games seriously, and there's no reason to prevent people from getting (appropriately) upset when something bad happens.

It's that modifier, though. "Appropriately." I'm not going to take issue with obscenities (or grammar). This is objectively a bad way to play games. Because, of course, you can't earn points if you don't win. And regardless of how bad (or new) someone is to the game, it's almost always better to have an additional player on the field trying to help you win. It's bad strategy and tactics to just heap abuse on poor players — a fact the game understands, which is why one of the preset communication options is "No problem."

It all essentially comes down to treating other humans as humans. I'm not casting broad aspersions about gamers, teenagers or even teenage gamers. Just a note that digitizing all interactions seems to have the broad effect of dehumanizing interactions, unless specific tactics are employed.

I don't know how to educate these people — I'm just someone flying a car around in a videogame. But I made my attempt. After the reprimand for complimenting the guy on his shot, I decided to help the only way I could: I chased down an errant shot by the other team and knocked it in our goal in overtime.

My girlfriend says it was a little petulant — I disagree, but not too strenuously. I broadcasted a message after the shot: "No matter how bad your teammates are, it's better to have them then not."

Is the guy going to change his actions? Probably not. But at least there was some negative reinforcement (losing ranking points). Maybe next time he'll at least keep his frustrations to himself. That, in my books, counts as a win.

Election night is always tense in a newsroom - even when, as the case with the Pennysylvania governor's race, the outcome isn't in doubt, there are still so many moving parts and so many things that can change. Whether it's a late-reporting county/precinct or trying to design a front page you've been thinking about for weeks, there's always something that can go wrong. That's why, this year, I tried to prep my part of the election coverage with as few manual moving parts as possible. Though (as ever) things did not go according to plan, it definitely provided a glimpse at how things might run — more smoothly — in the future.

I set out in the middle of October with two aspects of the coverage. The first, live election results, was something I've been in charge of since the first election after I arrived at the York Daily Record in 2012. I've always used some combination of Google Docs (the multi-user editing is crucial for this part, since our results are always scattered around various web pages and rarely scrape-able) and PHP to display the results, but this year I had my GElex framework to start from (even if I modified it heavily and now probably need to rewrite it for another release).

The results actually went incredibly smoothly and (as far as I know) encountered no problems. Everything showed up the way it was supposed to, we had no downtime and the interface is as easy I can conceivably make it. You can take a gander at the public-facing page here, and the Sheet itself here. The one big improvement I made this time around was on embeds. Though there's always been the ability to embed the full results (example), this year — thanks to the move to separate sheets per race — it was possible to do so on a race-by-race basis.

This helps especially in consideration with our print flow, which has always been that election stories get written so the exact vote totals can be inserted later via a breakout box. By embedding the vote totals into the story, this meant we didn't have to go back in and manually add them on the web.

The governor's race stole pretty much all the of the headlines (/front pages) in York County owing to its status as Tom Wolf's home county. For us, this meant we'd be doing twice as many live maps as usual. The county-by-county heat map is relatively cliché as political indicators go, but it's still a nice way to visually represent a state's voters.

Since he's a native, we also decided this year to include a map of just York County, coding the various boroughs and townships according to their gubernatorial preferences. My first concern was online — we've done both of those maps in print before, so worse case scenario we'd be coloring them in Illustrator before sending them to press.

I wanted interactivity, fidelity, reusability and (if at all possible) automation in my maps. When it came to reusability and fidelity, SVG emerged as the clear front-runner. It's supported in most major browsers (older flavors of IE excepted, of course), on mobile and scales well.

The other options (Raphael, etc.) locked us down paths I wasn't really comfortable with looking ahead. I don't want to be reliant on Sencha Labs to a) keep developing it and b) keep it free when it comes to things like elections and maps. I would have been perfectly fine with a Fusion Table or the like, but I also wanted to look at something that could be used for things other than geocoded data if the need arose.

Manipulating SVGs isn't terribly difficult ... sometimes. If the SVG code is directly injected into the page (I used PHP includes), it's manipulable using the normal document DOM. If you're including it as an external file (the way most probably would), there are options like JQuery SVG (which hadn't been updated in TWO YEARS until he updated it less than a week before the election, or too late for me to use) or this method (which I was unable to get to work). (Again, I just cheated and put it directly on the page.)

Manipulating fills and strokes with plain colors is fairly easy using jQuery, just change the attributes and include CSS transitions for animations. The problems arises when you try to do patterns, which are much different.

I wrote a tiny jQuery plugin (pluglet?) called SVGLite to assist with this, which you can read more about here. When backfilling older browsers, I figured the easiest thing to do was serve up PNG images of the files as they existed. Using everyone's favorite PHP library for ImageMagick, imagick, this was trivial. Simply running a few PREG_REPLACEs on the SVG file before serving it to Imagick helped me get the colors I needed.

It turns out there aren't a lot of free options for scraping data live, and as I've mentioned before, free is pretty much my budget for these sorts of things. But there is one. Import.io, which has the classic engineer's design problem of making things more difficult by trying to make them easier, turned out to be just what we needed when it came to pulling down governor's data.

Working off the results site for each county, I set up a scraper API that trolled all 67 pages and compiled the data for Wolf and Corbett. This was then downloaded into a JSON file that was served to the live Javascript and PHP/ImageMagick/PNG maps. Given that I didn't want to abuse the election results server (or melt ours), I built a small dashboard that allowed me to control when to re-scrape everything.

This part actually went almost as well as the live results, with one MASSIVE EXCEPTION I'll get to after the next part. The boroughs/townships data presented its own problems, in the form of only being released by PDF.

Now, running data analysis on a PDF is not terribly difficult — if you're not time-constrained, I'd definitely recommend looking into Tabula, which did an excellent job of parsing my test data tables (2013 elections), as well as the final sheet when it was all said and done.

Unfortunately, processing each one took about 45 minutes, which wasn't really quick enough for what we needed. So we turned to the journalist's Mechanical Turk: freelancers and staff. Thanks to the blood, tears, sweat and math of Sam Dellinger, Kara Eberle and Angie Mason, we were able to convert a static PDF of numbers into this every 20 minutes or so.

It's always a good idea to test your code — and I did. I swear.

My problem did not lie in a lack of testing, but rather a lack of testing using real numbers or real data. For readability purposes, the election results data numbers are formatted with a comma separating every 3 numbers, much in the way numbers always are in non-financial or -computer contexts (e.g., 1,000, 3,334,332).

UNFORTUNATELY, when I did all my testing, none of the numbers I used went above 1,000. Even when I was scraping the test data the counties were putting up to test their election results uploading capabilities, the numbers never went above 500 or so — or, if they did, they were tied (1,300 for Wolf, 1,300 for Corbett).

The problem lies in how the scraper worked. It was pulling all of the data as a string, because it didn't know (or care) that they were votes. Thus, it wasn't 83000, it was '83,000'. That's fine for display purposes, but it's murder on mathematical operations.

About an hour after our first results, the ever-intrepid and knowledgeable Joan Concilio pointed out that my individual county numbers were far too low - like, single or double digits, when the total vote count was somewhere north of 200,000. After walking all of my numbers back to import.io, I realized that I needed to be removing the commas and getting the intVal() (or parseInt(), where appropriate).

(I also originally intended to provide the agate data using the same method, but the time it took to quash the number bug meant it was safer/wiser to go with the AP's data.)

Conclusion:

1. Always test your data.

2. Always make sure your data matches the type you're looking for.

3. Sometimes the overhead of statically typed languages is worth the trouble.

Overall, everything went fairly well, with the exception of the aforementioned bug report (which also made us double- and triple-check the print graphic). The advantage of SVG, aside from its digital flexibility, was that after a quick Save-As and conversion in Illustrator, we had a working print file ready to go.

Another election, in the books.

As I've mentioned before, we're moving away from Caspio as our database provider to the extent that it makes sense (not out of utility, it's a function of cost). While we've managed to get some things migrated over, one of the biggest stumbling blocks are the things we use Caspio for the most — simple databases that need to be viewable and searchable online.

We have a number of semi-complex databases (read: more than a single-sheet XLS file) that we're not moving anytime soon (deed transfers database, among others, simply because of how we ingest the data), but there are a number that are little more than spreadsheets that we need to be able to view and search.

We investigated a number of vendor alternatives, but most featured pricing problems similar to Caspio, or had records limits absurdly lower than what we need. (Example: One such service offered 100,000 rows of data for $149/month. For comparison, one of our more popular databases, listing Pennsylvania teachers' salaries, has well over 2 million rows alone.) So, once again, Project Time™.

There is one thing that any aspiring programmer must realize when they set out to replace a tool: YOU CAN'T REPLACE A TOOL AT THE HEART OF A MULTI-MILLION DOLLAR CORPORATION ON YOUR OWN. I knew this academically but, as is often the case when setting out on these adventures, my brain chose to heed that advice only when it was convenient to do so.

I often live by the mantra, "If someone else can do it, that means it's possible." It works well something like 75 percent of the time — it prevents me from feeling daunted when facing large projects, but it can be turned around as well.

My favorite caveat is, "Technically, I could build you a reasonable facsimile of Facebook — it just wouldn't be as good, fast or as useful as the real thing."

It's true in that somebody built Facebook, but (more accurately) thousands of somebodies built Facebook. It's doable, it's just not feasible for one person to replicate it completely on their own.

That being said, Past Me was convinced it couldn't be THAT difficult to take a spreadsheet and present it online, despite the fact that people routinely pay up to and including hundreds/thousands of dollars per month to companies to be able do exactly that.

Ah, hubris.

The first priority involved figuring out how to store the data. The reason the York Daily Record likes Caspio so much is not just its versatility and usefulness, it's how easy it is to use. Caspio spent a lot of time and money into figuring out an interface that, while not everyone can use it and even fewer can take full advantage of all its features, it's easy enough that most people can do basic things with little training. This actually posed the greatest challenge — the data needed to be able to be input and edited in such a way that your average reporter (think 35-year-old metro reporter, not 23-year-old working at The Verge) would be able to do so without having to email/call me every five minutes. That ruled traditional databases out right away. (Which is not to say that you can't build an edit-friendly MySQL frontend, but I didn't have that kind of build time for this project.)

The easiest and cheapest way forward seemed to be (as ever) through Google. Though I'm becoming more wary of Google Docs' live-editing capabilities, for the purpose of "storing data and being able to edit it directly," Sheets fit the bill.

Because our CMS does not allow for server-side code inclusion (another story for another time), inserting the data into articles needs to be accomplished via JavaScript drop-in. Since we're going to be building it in JS anyway (and I'm a firm believer on not doing the same work twice unless I forget to commit something to the repository), I figured we'd just use one codebase for both the widget version and the standalone.

After a little bit of searching (I got burned out going through a dozen different Caspio alternatives), I settled on DataTables as our jQuery plugin of choice.

Here's the part where I always have trouble when trying to relate the struggles of the average newspaper's newsroom to the more digital-focused newsrooms who have multiple app developers and coders on staff — most newspaper reporters do not have the coding ability beyond making a link or typing into the TinyMCE in WordPress.

You can get them to do things like a YouTube embed using a tag interface [Youtube: https://www.youtube.com/watch?v=jvqfEeuRhLY], but only after some heavy-duty brainwashing (and we still struggle with getting Excerpts right).

So while I and probably three or four in our newsroom have no problem using Quartz's excellent ChartBuilder, it's not something we can just send out to the general population with a "use this!" subject line and expect results.

While some might be content with a simple "Use DataTables!" and inserting some code to auto-activate the tables when people set them up properly, asking your average journalist to use JavaScript parameters is a fool's errand, and we're not even within driving distance of, "Oh yeah, and get your Sheet into JSON for DataTables to use."

Which is not to call them stupid — far from it. It's just that these are people who spent a bunch of time (and, likely, money) to learn how to write stories properly. Then they got to work anytime after 2005 and discovered that it wasn't enough — they have to learn Twitter, Facebook, an ever-increasing number of content managements systems and (oh yeah!) they still have to do it while writing their stories. All of this is doable, of course, but to ask them to learn HTML and JavaScript and every new thing someone invents (which even I have given up all hope of keeping up with; there are just too many new things out there) is simply untenable.

Thus, I consider it my number one job to make their jobs easier for them, not just give them something complicated they have to learn just because it does a new thing (or an old thing in a cooler/cheaper way).

For the first version, it's about as simple as can be. People work on their data using their own preferred Google accounts (work or personal), leaving them with a document they can play around with. Once they're to a point where they're ready to present the data to the public, we copy the data into a separate account. This has the advantage of a) keeping the data under our control, in case the reporter quits/leaves/dies/deletes their account, and b) allows the reporter to keep their own copy of the data with the fields they don't want shown to the public (internal notes, personally identifying information, that sort of thing). The reporter then grabs the sheet ID from the URL and puts it in the tool.

Assuming the data passes some very basic tests (every column has a header, only one header row, etc.), they're presented with a list of fields. Because our CMS frontend does not allow for responsive design, all our information lives in 600 pixel-wide boxes. So with a little help from jQuery Modal, I added some functionality to DataTables using the standard hidden fields that hides some columns in the standard presentation, but shows the entire entry's information in a modal if a row is clicked.

For version 1, search is pretty simple: If there's a field, it's searchable. We're hoping to expand on that in later iterations to not search certain fields, as well as create some method of specifically searching fields (as seen in this Caspio implementation). Users then add a title (shown only in the full version; we're assuming wherever the widget drop-in goes, there's already a headline on the article) and customized search text.

They're then taken back to the main screen, where they can find links to the full data page (like this, which we use for our mobile implementation (neither our apps nor our mobile site executes JavaScript, so we always have to include links to a place off our main domain for our mobile readers to view) as well as the drop-in widget code.

Eventually, we hope to add some things like the extended search functionality, a "download data" option and other enhancements. But for now, we feel like we have a tool for basic database work.