Technology Impact Cycle Tool

In this crash course we assume that transparency is good.

Technologies and the companies that make our technologies must be transparent. In this section, we'll cover 4 ways to assess how a technology or technology company deals with transparency.

• Shortcomings;
• Business model;
• Accessibility/communication;
• How the technology works.

Shortcomings

Is it clear what the technology's shortcomings are?

Is it possible to know beforehand if a technology has shortcomings? This is not always the case. Of course, when you read the manual or terms of use, most of the time you will find the limitations of technologies.  However, when they are very clearly mentioned (for example, on the packaging) it is often for a legal reason. For example, if you buy a floating device (let’s say a unicorn) the packaging and maybe the unicorn itself will probably say that it is not a life-saving device and that you should not use the inflatable unicorn to cross the Atlantic.

And yet, there will always be people who try anyway.

It’s highly Instagrammable, but not advisable.

Digital technology companies however are often less transparent about the shortcomings of their products. Let's look at some examples:

• Is it clear that your sleeptracker is not really accurate?
• Can you see that your digital pedometer is often wrong about the number of steps?
• Does your smart brush tell you that it is not optimized for your teeth?
• Is a weather app honest about the trustworthiness of their rain radar?

Are those shortcomings easy to find? Or are they hidden somewehere in the terms of use?

Of course, we understand that it is not wise for companies that make digital technologies to advertise what the limitations of their products are. After all, a detergent manufacturer will always say that its detergent is the very best. Whiter than white!

However, digital technology is becoming increasingly intimate. Technology watches you while you sleep, it counts your heart beating, counts your steps, looks at what you weigh, what your temperature is. Technology sometimes even tries to determine how you feel. And with that intimacy comes a responsibility to be as transparent as possible about the limitations of the product.

So that a good choice can be made by, for example, a user.

Quick questions: Do you really know what the shortcomings of technology you are using are? Can you find them?

Business model

How does the technology company make money?

Is it clear what the technology company's business model is? Let's say, for example, there is a technology company that sells smart vacuum cleaners. Does the company make money by selling expensive vacuum cleaners or is the company also making money on collecting data? Would you like to know that the vacuum cleaner is mapping your living room, recognizing your furniture and your usage of the vacuum cleaner. And that this data, according to the terms of use, can be sold to 'third parties.' If you would have known that, would you have made another purchasing decision?

The data sucking vacuum cleaner is only an example. However, today more and more (technology) companies understand that real money can be made by collecting data, and they are not always transparent about that, so being aware of the business model is very important. Ask yourself: how does the company make money? Who is the (real) customer?

In crash course two we asked these questions regarding the social media companies.  

Another important question is: where is the money coming from?

For example, Uber has never been profitable. Uber's strategy is to conquer markets, by offering lower prices than the competitor, but they can only do that, because they are supported by money from venture capitalists. If you understand that, you can decide for yourself whether you want to use an Uber. 

Quick question: Do you think (technology) companies in general are transparent about their business model?

Accessibility/Communication

Is it easy to talk to someone from the technology company?

Is it easy to ask questions? Is it easy to find more information? Can you file a complaint? Maybe call someone? A helpdesk? Do you get a reply? Is a real person answering the phone?

It is hard to communicate with a lot of  (big) technology. Some companies are unreachable even for governments, let alone private persons. If your account is deleted on YouTube. If you do not understand why your LinkedIn messages get so few views? If you are banned from Twitter, who are you going to call?

(Logo from Ghostbusters)

Ghostbusters?

Dark patterns

Dark patterns are the opposite of transparency.

Dark patterns are tricks used in websites and apps that make you do things that you didn't mean to, like buying or signing up for something. There is a great site on it: darkpatterns.org (also see, additional materials). The purpose of darkpatterns.org is to spread awareness and to shame companies that use them.

When you use websites and apps, you don’t read every word on every page - you skim read and make assumptions. If a company wants to trick you into doing something, they can take advantage of this by making a page look like it is saying one thing when it is in fact saying another. You can defend yourself by learning about dark patterns.

There are many dark patterns. Here are two examples:

• Roach Motel. You get into a situation very easily, but then you find it is hard to get out of it (e.g. a premium subscription).
• Privacy Zuckering. You are tricked into publicly sharing more information about yourself than you really intended to. Named after Facebook CEO Mark Zuckerberg.

Since darkpatterns.org was founded, the number of dark patterns online are growing even harder. 

How does it work?

Is the technology company transparent about how the technology works or is programmed?

Why do you see that certain post in Facebook? Why is Tinder showing you that girl or boy? Why is Uber sending you that driver? Why does your FitBit tell you that you need to sleep more? Why does Booking.com tell me that 6 people are looking at the same room? Is that really true? Is there really only one room available anymore?

Shouldn't our technology be honest and transparent about the way it works?

Often, that is not the case.

And something new is at play. More and more, technology companies no longer even know exactly how their own technology works. Does Google still know why YouTube is recommending certain videos? Does Facebook understand their own algorithms that are selecting the newsfeeds? Does anybody know exactlly why that app thinks that birthmark on your arm is skin cancer? Or why the camera recognizes your face?

Advanced artificial intelligence becomes more and more a black box. Too complicated to understand. This is a very important development. What if in a few years advanced AI will decide that you are not fit for a job? Or for an education? Or a health insurance? What if the computer says NO or YES, but nobody knows why exactly anymore? What if new AI – systems need to be opaque to be accurate? What if designing for transparency leads to worse results?

What if there is a trade-off between accuracy and transparency?

Of course, it is important for technology companies to be transparent about the shortcomings of their technology and about their business model. Of course, it should be easy to communicate with a technology company. But when we talk about transparency, advanced artificial intelligence is by far the most important topic. How do we relate to these systems that are more and more a black box? 

That is the subject of the remainder of this course. Starting with section two, an introduction to AI.

Take aways from section one:

• Technology companies should be clear about the shortcomings of the technology;
• And about the underlying business model
• And they can make sure that they are easy to reach;
• But can they still be transparent about how their technology works when they are using advanced AI?