Excellence in Software

The “Excellence in Software” score that all software should strive for is “AAAA”. This is tough. Only the very best will achieve it.

The acceptance criteria to use depends upon the type of software under development. For many apps and websites the criteria would be close to the following, with items grouped into four sections: accuracy, usability, safety and compliance. Consider these to be a good starting point.

Accuracy. These are the classic measures of bug-free software.

1. Does the software consistently produce correct results?
2. Does the software gracefully handle situations with missing, invalid or extreme input data?
3. Does the software faithfully save and restore user data?
4. Do sorting routines correctly work with foreign language data?
5. Do search routines find and filter results in a way the user expects?
6. Do summary records always reflect the true sum of the details?

Usability. These are measures of how well the software accommodates users with different levels of sophistication; users having older hardware; and users with less-than 20/20 health.

1. Does the software work on every device and browser used by the top 98% of users?
2. Does the user interface have readily available hints and prompts to help new users understand and operate each aspect of the software?
3. Are form validation messages given to the user in a place and manner that is obvious?
4. Do background colors and text colors have sufficient contrast to be readable?
5. Does informational feedback still work for people with protanopia (red-green color blindness)?
6. Do web components and generic HTML tags have appropriate WAI-ARIA roles assigned?
7. Does the software accommodate screen readers by numbering HTML tabindex attributes in a most-significant to least-significant order?

Safety. These are measures of how well the user’s data is protected from accidental loss, bad actors, and catastrophic failures.

1. Does the data backup schedule provide a stable snapshot of everything at hourly, daily, and weekly intervals?
2. Can the data restoration process be completed within two hours?
3. Is the catastrophic data storage facility in a separate location from the online data?
4. Are user credentials properly encrypted during storage and during network transmission?
5. Is multi-factor authentication triggered when access is requested from a new location?
6. Are web servers configured to use up-to-date TLS protocols?
7. Are the website’s Content Security Policy (CSP), Feature Policy, and Referrer Policy properly set up to detect and report breaches?
8. Are sequential records protected from brute force and replay attacks?
9. Are user interface redress attacks prevented with proper use of the frame-ancestors CSP header directive?
10. Is user input sanitized to catch cross-site scripting injections when saving data to the database?
11. Are financial transactions protected with the use of HMAC-based cross-site request forgery tokens?
12. Have all files been scanned for viruses before deployment?
13. Are DevOps’ server credentials stored separately and outside project repositories?
14. Does the project’s testing protocol include explicit checks for common data breaches?
15. Are web server access logs regularly monitored for abnormal conditions?

Compliance. These are measures of the software’s adherence to industry standards and civil law.

1. Do the website’s documents use HTML and CSS in a way that passes W3C validators?
2. Do browsers evaluate the website’s JavaScript without generating console warnings or errors?
3. Does the website have a data privacy policy that complies with the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA)?
4. Does the software’s eCommerce process comply with Payment Card Industry Data Security Standards (PCI DSS)?
5. Does the software comply with the Access to Information and Communication Technology (ICT) Section 508 standards for government sites?
6. Does the software adhere to the terms and conditions of each third-party software license it uses?
7. Are third-party software libraries that are distributed under GNU or MIT licenses properly acknowledged on a public-facing document?
8. Does DevOps have a written schedule of expiration and renewal dates for all limited term licenses?
9. Has the company established a vulnerability disclosure policy with a way for outside security researchers and bounty hunters to privately provide notification of issues?

Excerpted from Excellence in Software originally published in Level Up Coding.