*Forward the Original Title ‘“I support it only if it’s open source” should be a more common viewpoint’

One concern that we often hear about certain kinds of radical technologies is the possibility that they will exacerbate power inequalities because they will inevitably be available only to the wealthy and powerful.

Here is a quote from someone concerned about the consequences of life extension:

“Are some people going to be left behind? Are we going to make society far more unequal than it is now?” he asked. Tuljapurkar predicted that the lifespan boom will be confined to wealthy countries, where citizens can afford anti-aging technology and governments can afford to sponsor scientific research. This disparity complicates the current debate over access to healthcare, as the rich become increasingly distanced from the poor, not only in quality but length of life.

“Big pharmaceutical companies have a well-established track record of being very difficult when it comes to making things available to those who can’t pay for them,” he said.

If anti-aging technologies are distributed in the unchecked free market, “it’s entirely likely to me that we’ll wind up with permanent global underclasses, countries that will get locked into today’s mortality conditions,” Tuljapurkar said … “If that happens, you get negative feedback, a vicious circle. Those countries that get locked out stay locked out.”

Here are some equally strong words from an article worrying about the consequences of human genetic enhancement:

Early this month, scientists announced that they had edited genes in a human embryo to remove a disease-causing mutation. The work was astounding and the answer to prayers of many parents. Who wouldn’t want a chance to prevent what would now be needless suffering by their children?

But that wouldn’t be the end of it. Many parents would want to ensure their children had the best of advantages through genetic improvement. Those with means could obtain them. With the ability comes ethical questions beyond the ultimate safety of such techniques. Expense of procedures will produce scarcity and aggravate income inequality that already continues to grow.

And similar viewpoints in other technology areas:

• Digital technology in general: https://www.amnestyusa.org/issues/technology/technology-and-inequality/
• Space travel: https://oilprice.com/Energy/Energy-General/What-Does-Billionaires-Dominating-Space-Travel-Mean-for-the-World.html

• Solar geoengineering: https://www.cambridge.org/core/journals/global-sustainability/article/hidden-injustices-of-advancing-solar-geoengineering-research/F61C5DCBCA02E18F66CAC7E45CC76C57

  You can find this theme in many criticisms of new technology. A somewhat related, but importantly different, theme is that of technological products being used as a vehicle for data collection, vendor lock-in, deliberately hidden side effects (eg. modern vaccines have been criticized this way), and other forms of abuse. Newer technologies tend to create more opportunities to give someone a thing without giving them the rights to a thing or the full information about a thing, and so from this lens too older technologies often seem safer. This is also a form of technology strengthening the powerful at others’ expense, but it’s an issue of manufacturer-against-user power projection through the technology rather than the concern in the previous examples, which is inequality of access.

  I personally am very pro-technology, and if it were a binary option of “go further” vs “stay where we are”, I would gladly push forward on everything except for a very small list (eg. gain of function research, weapons and superintelligent AI) despite the risks. This is because on the whole the benefits - much longer and healthier lives, a more prosperous and society, preserving more human relevance in an era of rapidly improving AI, maintaining cultural continuity through older generations surviving as people and not just as memoirs in history books - are much larger than the downsides (which often end up overrated).

  But what if I put myself into the shoes of someone who is either less sunny on the positive implications, or more concerned that powerful people will use new technologies to perpetuate their economic dominance and exert control, or both? For example, I already feel that way toward “smart home” stuff - the benefit of being able to talk to the light bulb is outweighed by not wanting my personal life to be streamed to Google or Apple. If I had more pessimistic assumptions, I could also see myself feeling that way toward some media technologies: if they enable powerful people to broadcast messages more effectively than everyone else, then they can be used to exert control and drown out others, and for many such technologies the gains from us having better information or better entertainment are not large enough to compensate for the way they reallocate power.

Open source as the third way

One viewpoint that I think is heavily under-valued in these situations is: supporting a technology being developed only if it’s open source.

There’s a very plausible case that open source accelerates progress: it makes it much easier for people to build on each other’s innovations. There’s also a very plausible case that requiring open source decelerates progress: it prevents people from using a large class of potential strategies to make money. But the most interesting consequences of open source as those that pull in directions unrelated to the “faster vs slower progress” axis:

• Open source improves equality of access: if something is open source, it is naturally accessible to anyone in any country. For physical goods and services, people still have to pay marginal (per-item) costs, but in a large number of cases, prices of proprietary products are high because the fixed costs (eg. NRE) of coming up with the thing are too high to invite more competition, and so the marginal costs are often quite cheap (eg. this is definitely true in pharma).
• Open source improves equality of access to being a producer. One criticism of giving people free access to end products (even unquestionably good ones, like healthcare) is that it doesn’t help those people gain skills and experience and climb up the global economy into prosperity, which is the best truly reliable guarantor of lasting access to high-quality life (see eg. Magatte Wade complaining about this regarding aid to Africa). Open source is not like this: it is fundamentally about enabling people anywhere in the world to be producers at all parts of the supply chain, and not just consumers.
• Open source improves verifiability: if something is open source (which ideally should include not just the output, but also the process used to come up with it, make parameter choices, etc), then it’s much easier to verify that what you’re getting is what the provider claims you’re getting, and for third parties to do research to identify hidden downsides.

• Open source removes opportunities for vendor lock-in. If something is open source, then the manufacturer cannot render it useless by remotely removing features, or simply by going bankrupt (eg. see concerns about highly computerized/networked cars no longer working if the manufacturer shuts down). You always have the right to repair things yourself (or by asking a different provider).

  We can analyze this from the perspective of some of the more radical technologies that I listed near the beginning of the article:

• If we have proprietary life extension technology, then it may be only accessible to billionaires and political leaders (I personally expect that this technology will drop in price quickly, but your opinion on this may be more skeptical than mine). But if it’s open source, then anyone can go and use it and offer it to others cheaply.

• If we have proprietary human genetic enhancement, then it may be only accessible to billionaires and political leaders, creating an overclass. (Again, I personally think such tech will diffuse, but there will definitely be some delta between what the wealthiest get and what the average person gets). But if it’s open source, the delta between what the well-connected and powerful get and what everyone else gets will be much smaller.
• For any biotech in general, an open-science safety testing ecosystem may well be more effective and honest than a company endorsing the safety of its own product and getting rubber-stamped by a pliant regulator.
• If only a few people can go to space, depending on how politics goes there’s some chance one of them will take an entire planet or moon for themselves. If the technology is more widely distributed, they will have less opportunity to do so.

• If your smart car is open source, then you can verify that the manufacturer is not spying on you, and you are not dependent on the manufacturer to be able to keep using your car.

  We can sum the argument up in a chart:

Note that the “build it only if it’s open source” bubble is wider, reflecting larger uncertainty in just how much progress open source will lead and just how many power concentration risks it will prevent. But even still, on average it’s a good deal in a large variety of situations.

Open source and misuse risk

One major argument against open sourcing powerful technologies that sometimes gets brought up is the risk of zero-sum behavior and non-hierarchical forms of abuse. Giving everyone nukes would certainly end nuke inequality (which is a real problem; we see multiple instances of powerful states using asymmetry of access to nukes to bully others as we speak), but it would also almost certainly lead to billions of deaths. To give an example of negative social consequences without deliberate harm, giving everyone access to plastic surgery may well lead to a zero-sum competitive game where everyone spends a lot of resources and even takes health risks to look more beautiful than everyone else, but at the end we all get used to the higher levels of beauty and society ends up not really being much better. Some forms of biotech could end up having these kinds of effects on a larger scale. Many technologies (in fact, lots of biotech) are in between these two extremes.

This is a valid argument for wanting to go the opposite way: “I support it only if it’s carefully controlled by trustworthy gatekeepers”. Gatekeepers could allow positive use cases of a technology while keeping out negative use cases. Gatekeepers could even be given a public mandate to ensure non-discriminatory access to everyone who does not break certain rules. However, I have a strong default skepticism of this approach. The biggest reason why is that I am generally skeptical that, especially in the modern world, trustworthy gatekeepers truly exist. Many of the most zero-sum and risky use cases of technology are military use cases, and militaries have a poor history of constraining themselves.

A good example is the Soviet bioweapons program:

Given his restraint with regard to SDI and nuclear weapons, Gorbachev’s actions related to the Soviet’s illicit germ weapons program are puzzling, noted Hoffman.

When Gorbachev came to power in 1985, the Soviet Union had an extensive biological weapons program initiated by Brezhnev, despite being a signatory of the Biological Weapons Convention. In addition to anthrax, the Soviets also were working on smallpox, plague and tularemia, but their intentions and targets for such weapons are not clear.

“Kateyev’s papers showed there were multiple Central Committee resolutions about the biowarfare program issued in the mid- to late-80s. It’s hard to believe these were all signed and issued without Gorbachev’s knowledge,” Hoffman said.

“There’s even a May 1990 memo to Gorbachev about the biological weapons program – a memo that still didn’t tell the whole story. The Soviets misled the world and they misled their own leaders.

Oh, and see this link arguing that this bioweapons program may have ended up being made available to other countries (!!) after the Soviet collapse.

Other countries have large mistakes to answer for themselves. I need not link to everything that has been uncovered regarding many countries’ participation in gain-of-function research and the risks that it implies (this book is good though). In the realm of digital software (eg. finance), the history of weaponized interdependence shows how what was meant as abuse prevention easily slides into one-sided power projection by the operator.

This is another weakness of gatekeepers: by default, they will be controlled by national governments, and these countries’ political systems may well have an incentive to ensure equality of access within the country, but there is no powerful entity with a mandate to ensure equality of access between countries.

To be clear, I am not saying “the gatekeepers are bad too, so let’s have a free for all” (at least, not for gain-of-function research). Rather, I’m saying two things:

• If something has enough “all-against-all abuse” risks that you would only feel comfortable seeing it done in a locked-down way with centralized gatekeepers, consider that the correct solution may be not doing it at all (and investing in alternative technologies with better risk profiles)
• If something has enough “power dynamics” risks that you currently do not feel comfortable seeing it done at all, consider that the correct solution may be doing it, but doing it open source so that everyone has a fair chance to understand and participate.

Note also that “open source” does not imply “free for all”. For example, I would favor geoengineering being done in an open-source and open-science way. But this is not the same as “anyone can go redirect any rivers and sprinkle what they want into the atmosphere”, and it will not lead to that in practice: laws and international diplomacy exist, and such actions are easy to detect making any agreements quite enforceable. The value of openness is (i) ensuring that it’s more democratized (eg. usable by many countries instead of just one), and (ii) increasing the accessibility of information, so people can more effectively form their own judgements of whether or not what is being done is effective and safe.

Most fundamentally, I see open source as the strongest possible Schelling point) for how technology can be done with less risk of concentrating wealth and power and asymmetric information. One can try to construct more clever institutions that try to split apart the beneficial and negative use cases of a technology, but in the modern chaotic world, the approach that is most likely to stick is a very easily publicly-understandable guarantee that things are happening in the open and anyone can go and understand what’s going on and participate.

In many cases, these concerns less important than the extreme value of making technology go faster (or, in a few cases, the importance of slowing it down as much as possible, until either countermeasures or alternative ways of achieving the same goal are available). On the margin, however, the third option - focusing less on rate of progress, and more on style of progress, and using a norm of expecting open source as an easily legible lever to push things in a better direction - is an underrated approach.

Disclaimer:

1. This article is reprinted from [vitalik]. All copyrights belong to the original author [vitalik]. If there are objections to this reprint, please contact the Gate Learn team, and they will handle it promptly.
2. Liability Disclaimer: The views and opinions expressed in this article are solely those of the author and do not constitute any investment advice.
3. Translations of the article into other languages are done by the Gate Learn team. Unless mentioned, copying, distributing, or plagiarizing the translated articles is prohibited.