DEV Community

Discussion on: Papers We Love - Reflecting on Trusting Trust

Collapse
 
rrampage profile image
Raunak Ramakrishnan

Thanks once again for starting this! This article almost disappeared in my feed. I think you should add a note in the post to follow #pwl tag so that we will not miss out :)

The first time I read the paper (almost 8 years back) I was impressed by Ken Thompson's ingenuity and deviousness in the construction of the compiler and how it tweaked the login program and the compiler binary itself. It was the first security related paper I had read.

Regarding ethics, I unfortunately think we are doomed to repeat our failings from similar advances in chemistry (explosives), physics (atomic energy). Technology always seems to advance at a pace faster than our legal and social norms keep up with.

With the advent of machine learning and increasingly pervasive data collection, we are in territory where humans do not understand why the algorithms make certain predictions. Many such algorithms are merely amplifying our inherent bias either due to faulty modeling or biased training data. Cathy O Neil's Weapons of Math destruction is a good overview of the same.

Collapse
 
rhymes profile image
rhymes

With the advent of machine learning and increasingly pervasive data collection, we are in territory where humans do not understand why the algorithms make certain predictions.

Agreed. I don't think there's a chance to be "stopping" anything here. We don't understand what we're doing, why should we stop? I hope, after a period of "stupid things done by very smart people", we'll start thinking and talking more before implementing things. In the meantime I have faith in the new batches of programmers :-)

There's a difference between "can" and "should" :D