A brilliant write up and I totally get your frustration. Obscure algorithms often make us wonder about bias I guess - are they skipping those songs for a reason? Financial? Their own moral code... You can make a conspiracy out of everything.
It's not just random though. For instance, our application tries to present records in a sensible order for users. The problem is that the inputs to what makes an ideal order are simple to propose but have "unintended consequences".
So in our case, we provide a service where users can prove their credentials to perform certain hazardous and semi-hazardous actions and demonstrate that they have the right levels of insurance. There is a legal requirement in many jurisdictions to vet external parties before they have access to a worksite.
What order do we put the results in? We have a Google-like search, so I can search for "Bob Plumber Air Con" near some location and it will respond in order. Now with geography, it will use a basic drive time calculation to order the results, and then it will score based on where matches were found and if they were exact matches or "sounds like" ones.
We also want to rank based on factors like "is their insurance info up to date" and then you have salespeople wanting to rank people higher if they are new or up for renewal. Those kinds of moral decisions have to be removed because they just pollute the whole thing with no meaningful purpose for the user. Even without them you can end up in some strange result sets that the user thinks are totally broken, even though they are absolutely working fine.
I guess the moral of that is: if the user thinks it's broken, it's broken. If two or more users would have conflicting views you have to give them the ability to solve the problem in a way that works for them.
Obscure algorithms often make us wonder about bias I guess - are they skipping those songs for a reason? Financial? Their own moral code... You can make a conspiracy out of everything.
Exactly. I went through multiple mental cycles trying to determine if there was some kinda underlying motive to the rigged shuffling. In the end, it doesn't really matter whether there is or isn't something nefarious. The critical point is that obfuscated processes can foster conspiracy theories.
Those kinds of moral decisions have to be removed because they just pollute the whole thing with no meaningful purpose for the user.
I've been through a similar scenario. I worked for years on applicant tracking systems. We had this fancy comparison engine that would try to do a "smart match" between a job description and the applicants' resumes. Then it would assign a score and sort the applicants from best-matched (theoretically) to least-matched.
For some of our big corporate clients, we had to disable this feature. The reason was that, to comply with federal hiring statutes and liability concerns, the companies had to keep records of tangible qualifications that had led them to choose one candidate over another (as a counterpoint to the idea that they may have hired the applicant because they were male, or white, or whatever).
But if they were sorting candidates by our magic (meaning: obtuse) algorithm, there was no way that they could satisfy the audit requirement. In fact, their lawyers informed them that merely having this feature enabled in our app represented a liability to their company. Because if it was enabled, and some hiring manager sorted by that "magic" number (whether it was by accident or on purpose), that could theoretically bias the hiring manager in favor of a criterion that they could not define.
I guess the moral of that is: if the user thinks it's broken, it's broken. If two or more users would have conflicting views you have to give them the ability to solve the problem in a way that works for them.
Absolutely. That's definitely my biggest frustration with the Spotify anecdote. I couldn't care less that they wanted to create their own fancy "better" shuffling algorithm. But when they did that, they removed the basic algorithm - the one that did real shuffling.
There's an overall air of condescension there that I've actually seen in many other companies/apps. It's this mindset that says, "We've decided that this is the best approach for you. You're welcome."
To me, it's particularly galling on the shuffle algorithm. They changed it because so many people didn't understand/appreciate what was happening when it was just a simple shuffle. I get that. But I do understand/appreciate it. And I definitely want to have the option for a real shuffle.
Some comments have been hidden by the post's author - find out more
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
A brilliant write up and I totally get your frustration. Obscure algorithms often make us wonder about bias I guess - are they skipping those songs for a reason? Financial? Their own moral code... You can make a conspiracy out of everything.
It's not just random though. For instance, our application tries to present records in a sensible order for users. The problem is that the inputs to what makes an ideal order are simple to propose but have "unintended consequences".
So in our case, we provide a service where users can prove their credentials to perform certain hazardous and semi-hazardous actions and demonstrate that they have the right levels of insurance. There is a legal requirement in many jurisdictions to vet external parties before they have access to a worksite.
What order do we put the results in? We have a Google-like search, so I can search for "Bob Plumber Air Con" near some location and it will respond in order. Now with geography, it will use a basic drive time calculation to order the results, and then it will score based on where matches were found and if they were exact matches or "sounds like" ones.
We also want to rank based on factors like "is their insurance info up to date" and then you have salespeople wanting to rank people higher if they are new or up for renewal. Those kinds of moral decisions have to be removed because they just pollute the whole thing with no meaningful purpose for the user. Even without them you can end up in some strange result sets that the user thinks are totally broken, even though they are absolutely working fine.
I guess the moral of that is: if the user thinks it's broken, it's broken. If two or more users would have conflicting views you have to give them the ability to solve the problem in a way that works for them.
Exactly. I went through multiple mental cycles trying to determine if there was some kinda underlying motive to the rigged shuffling. In the end, it doesn't really matter whether there is or isn't something nefarious. The critical point is that obfuscated processes can foster conspiracy theories.
I've been through a similar scenario. I worked for years on applicant tracking systems. We had this fancy comparison engine that would try to do a "smart match" between a job description and the applicants' resumes. Then it would assign a score and sort the applicants from best-matched (theoretically) to least-matched.
For some of our big corporate clients, we had to disable this feature. The reason was that, to comply with federal hiring statutes and liability concerns, the companies had to keep records of tangible qualifications that had led them to choose one candidate over another (as a counterpoint to the idea that they may have hired the applicant because they were male, or white, or whatever).
But if they were sorting candidates by our magic (meaning: obtuse) algorithm, there was no way that they could satisfy the audit requirement. In fact, their lawyers informed them that merely having this feature enabled in our app represented a liability to their company. Because if it was enabled, and some hiring manager sorted by that "magic" number (whether it was by accident or on purpose), that could theoretically bias the hiring manager in favor of a criterion that they could not define.
Absolutely. That's definitely my biggest frustration with the Spotify anecdote. I couldn't care less that they wanted to create their own fancy "better" shuffling algorithm. But when they did that, they removed the basic algorithm - the one that did real shuffling.
There's an overall air of condescension there that I've actually seen in many other companies/apps. It's this mindset that says, "We've decided that this is the best approach for you. You're welcome."
To me, it's particularly galling on the shuffle algorithm. They changed it because so many people didn't understand/appreciate what was happening when it was just a simple shuffle. I get that. But I do understand/appreciate it. And I definitely want to have the option for a real shuffle.