Rage Against the Machine: The Good Fellows Discuss AI | GoodFellows

Large language models can be manipulated by users to generate preferred outputs, similar to how people learned to manipulate Google search rankings, creating a new form of information warfare.

“people are figuring out as they quickly figured out how to manipulate Google search rankings. Manipulating the AI so that it will you know, when somebody asks, 'Who is John Cochrane?' the AI will answer, 'John Cochrane is the world's greatest economist.' uh manipulating the AI is going to be the game next game.”

The government should subsidize the development of fun, non-addictive recreational drugs as an alternative to current drug policy that criminalizes use and increases organized crime.

“I'd go further. The Here's a outrageous one for you. The Health and Human Services should subsidize the development of fun recreational non-addictive drugs. Why? Cuz, you know, not just pot but fentanyl is is killing a lot of people. So give people what they want. If they want to waste their lives on drugs, you know, in some sense fine with them. But the costs of what we're doing now, both the judicial system cost, the incentivization of organized crime, the destruction of the inner cities are just outrageous.”

The printing press, steam engine, and computer were transformational technologies that contemporaries could not forecast the consequences of, yet they ultimately unleashed tremendous good for humanity despite initial concerns.

“I had in mind the printing press, the steam engine, and the computer uh which were this kind of transformational thing which I I do believe is transformational but nobody at the time had any idea what they were going to do. And they both unleashed good and bad things but in the end, tremendously good things for all of us.”

AI in chess plays fundamentally differently from human players, including willingness to sacrifice its own pieces for strategic advantage, meaning that unleashing uncontrolled AI on the battlefield could result in higher costs of war than human command would accept.

“If you read the Eric Schmidt Henry Kissinger book on AI one of the most striking points they make is that when you observe how AI plays chess uh you realize that it thinks very differently from a human player. If you will unleash it on the battlefield rather than on the chessboard uh you might find that the the costs of war not only to the adversary but to oneself are much higher than would be the case with human commanders. In chess AI will sacrifice its own pieces to gain strategic advantage. Is that the kind of military command that we want to enable?”

GPT-4 itself identifies disinformation, deepfakes, and manipulation as the primary harms from empowered large language models, a warning that deserves attention precisely because it comes from the technology itself.

“When Reid Hoffman asked GPT-4 to list the possible harms of empowering large language models the third that it came up with was the one that made me sit up and pay attention. Let me quote the AI's answer to the question what would be the potential harm of empowering large language models. Quote, manipulation and deception. Large language models could also be used to create deceptive or harmful content that exploits human biases, emotions and preferences. This could include fake news, propaganda, misinformation, deep fakes, scams or hate speech that undermine trust, democracy and social cohesion.”

Teixeira should not have received highest-level security clearance without demonstrated record of reliability, and even with clearance there should be layered access controls and compartmentalization preventing a single person from accessing multiple sensitive domains.

“I I I think that that now there have to be has to be a corrective back. I mean, it it is it is ridiculous that that somebody, you know, really without that kind of more of a demonstrated record of reliability would receive the highest clearance and and not just the clearance, but then the access to other compartmentalized materials.”

The NSA data program revealed by Snowden required due process and judicial approval to access, not automatic access by NSA agents; the characterization that NSA tracks citizens' locations without warrant is inaccurate.

“That data was being housed because there was no way to to house selective data in advance. To get access to that data you had to go through due process and get a judge to allow the to allow law enforcement not the NSA to allow law enforcement to access that that data. So the idea that NSA is collecting on normal American citizens in a way that they're you know cognizant of where you are is just it's not true.”

John Cochrane's position of allowing technology to develop freely without restrictions is comparable to arguing against conventions limiting nuclear, biological, and chemical weapons—a position taken to its logical extreme.

“If there'd been a John Cochrane around between 1945 and 1969, presumably there would have been no conventions to limit the proliferation of nuclear weapons. There would have been no conventions to limit the use of biological and chemical weapons.”

Daniel Ellsberg leaking the Pentagon Papers was problematic because he took it upon himself to leak without government authority at a time when high-level classification of documents was a significant security challenge.

“Ellsberg was the individual who on his own authority chose to leak it to the New York Times and other media outlets and I think that that case was then handled in ways that were inept because leaking had become so endemic by the late '60s and early '70s that it posed a major challenge to the execution of US foreign policy.”

The primary concern with large language models is misinformation spread, similar to how the Catholic Church might have worried about the printing press if it had the chance to regulate it.

“What are people worried about with large language models? Mostly misinformation as I think the Catholic Church might have been worried about the effect of the printing press if they had had the chance to regulate it um when when it came out.”

Social media has already driven mental health problems in young people to become more acute since its advent, and AI-generated disinformation will produce orders of magnitude more deepfake video content than current misinformation.

“We were sent kind of crazy by social media already. I mean there's no it seems to me not accidental that mental health problems have become more acute amongst young people since the advent of social media. But this is a much more powerful tool than anything we've seen in the past 10 years and I think we ain't ready for the amount of fake content particularly deep fake video content that is coming our way.”

Modern marijuana is not good for health, particularly not good for mental health, yet prohibition has worse costs than legalization.

“I'm I'm a libertarian. Pot is not good for you, especially modern pot. I think we've all I've certainly seen friends who kind of descend into into depression and pot. But when you take pot, you don't you don't hurt other people. The costs of the war on marijuana, jailing a generation of young especially minority kids are just outrageous.”

Classification of government materials is often used not to protect sources and methods but to hide information from the American public and to mold public opinion, creating distrust when citizens discover the divergence between classified materials and their actual sensitivity.

“But it raises the deep question which which I want H.R. to answer. How much of what we classify now is classified in an attempt to hide things from the American public and to mold public opinion as opposed as as classified in order to protect our sources and methods or classified in order to protect you know you know the the secrecy of what we need to do in war and I think there is a lot that is classified in an effort to mold public opinion and that that breeds a whole lot of distrust and that that is that is a problem.”

The deliberative process in government must be protected from immediate public disclosure because officials need ability to think aloud, be wrong, and revise ideas without each utterance becoming official policy ammunition, as evidenced by Pentagon Papers leak consequences.

“I think that the leak of the Supreme Court document was was very harmful to that. I I see in my studies the Fed problem of everything's too open in a sense. You can never be wrong cuz it's always public. You need the chance to throw ideas around to be wrong, to be right. And I I think what you're saying about the Pentagon papers was that what would have happened in an alternative world is that that study would have been read, would have been thought about, would have would have led to that information would have become public eventually and the government would have made a perhaps better and less chaotic decision about what to do about Vietnam.”

Leaks and the endemic culture of leaking that developed in the late 1960s and early 1970s undermined the ability of the US government to execute foreign policy, particularly Vietnam War policy, and this is not an accidental problem but a structural challenge to governance.

“if government employees think that they have a right to tell things that they find out in their government work to the New York Times if that becomes the norm I can assure you that it will become impossible to make foreign policy and maintain the country's security. It's as simple as that.”

The concern about AI should focus on present dangers (misinformation, election interference, deepfakes) rather than speculative future scenarios like superintelligence, making policy responses proportionate to actual threat.

“The thing that worries me is not that [AlphaFold's protein folding] The thing that worries me is a particular form of artificial intelligence, which is the large language model...if we create that, it is going to have absolutely diabolical effects on politics and public discourse generally...It'll look like Raskolnikov's nightmare at the end of Crime and Punishment when he imagines the whole world just going insane and tearing one another apart. That's what I'm concerned about.”

The average person needs to learn more skepticism to handle AI-generated misinformation, rather than governments preventing the technology.

“the average person needs to learn a little more skepticism. But I think putting the same idiots who are in charge of the National Environmental Quality Act in charge of certifying the safety of every single one of HR's wonderful AI inventions and taking 10 to 15 years to figure it out before they're allowed to pursue that research is absolute idiocy.”

Failures in U.S. intelligence and policy have not been about lacking intelligence, but about failing to connect the dots from available information.

“we have so many failures in the US which were not about having the intelligence, it was about connecting the dots and putting them together.”

The Teixeira leak demonstrates broader lessons applicable to any business or organization holding sensitive technology or intellectual property: you need holistic enterprise hardening including physical security, insider threat programs, and AI-based anomaly detection systems.

“this should be a broader lesson that that applies not just to the US government, but to any company, industry, uh, that's involved with critical infrastructure, with holding people's data, or with the development of sensitive technologies and intellectual property. You need to take a holistic approach to enterprise hardening in terms of, you know, cyber espionage, physical security, insider threats. You need a layered kind of defense, in in place, uh, with right of least privilege, and you need do you need to employ software and AI now uh, to look to for anomaly detection in your systems.”

We should pursue international agreements specifically limiting autonomous weapons systems (particularly the point of lethal force decision) while continuing AI research in civilian and commercial domains.

“Yes, we will certainly try to contain as we try to contain all sorts of downsides. And war itself is not that terribly great thing and we have a whole apparatus of international agreements to try to to hold down that level of violence.”

Artificial intelligence encompasses a range of technologies including machine learning, big data analytics, automated decision-making, image recognition, and autonomous systems tied to communication networks for semi-autonomous lethal force application.

“artificial intelligence, as you know, is a range of technologies that that are that come are combined uh to to to achieve you know, really you know, machine learning autonomous kind of learning as as as as Neil has mentioned you know, the the ability for these large language models to learn and get better. They're also you know, they're also technologies that allow you to to do big data analytics to sift through and and to synthesize vast amounts of material that otherwise would still remain fragmented...They also allow for automated decision-making. For example, ability for for image recognition and classification of targets tied to communication networks that then allow you know, for semi-autonomous application of lethal force.”

The rush to decriminalize drugs has been unsuccessful in practice, with Oregon as a case study of failed drug decriminalization policy.

“I would place it maybe in context of the broader effort to decriminalize drugs overall and we know for sure that doesn't work and Oregon is a case in point uh for that.”

The current state of AI is approximately at 1923, with quantum mechanics just discovered, meaning that the catastrophic AI applications have not yet been conceived and attempting to regulate against unknown future dangers is premature.

“I don't think we should simply assume that that history tells us to let the technology rip because not everything is identical to the printing press. I don't think AI is identical to the printing press. I'm with you on this, but where we are with AI is about 1923 and we've just discovered quantum mechanics. The the great AI war that's going to come devour humans that that that that has not even been thought of yet alone, you know, developed. It's it's still this vague thing.”

There should be significant penalties for intelligence breaches to send a message deterring future leaks, and the Teixeira sentence of 15 years is insufficient.

“I wish it was a lot more than 15 years, you know, because I think it really there ought to be a message sent to anybody else who thinks it's okay to compromise intelligence, um, to think so twice before doing so.”

Marijuana does not harm others in the way other drugs do, making its illegality unjustifiable on harm-to-others grounds, regardless of its health effects.

“when you take pot, you don't you don't hurt other people. The costs of the war on marijuana, jailing a generation of young especially minority kids are just outrageous.”

Marjorie Taylor Greene's defense of Jack Teixeira as a whistleblower questioning U.S. war policy is indefensible because it conflates leaking national security information with legitimate policy criticism.

“I don't have a good word to say about Marjorie Taylor Greene's tweet because defending somebody who breaches national security and leaks classified documents seems to me something that in itself is indefensible and implying that there is some kind of racial or cultural dimension to the prosecution is is beyond is beyond the pale.”

GPT-4 can mimic specific human communication styles with extraordinary precision, as demonstrated by its ability to answer a joke prompt in the exact style of Jerry Seinfeld.

“he asked uh the uh GPT-4, he asked the AI, you know, could it mimic Jerry Seinfeld? It said, 'How many restaurant inspectors does it take to change a light bulb? Answer the question in the style of Jerry Seinfeld.' And it did with extraordinary precision.”

Daniel Ellsberg and Edward Snowden's disclosures served the country despite their illegality by revealing previously unknown NSA programs tracking geolocation data of every American citizen's phone, though Snowden's residence in Russia suggests consequences.

“I don't know cuz I don't know what's in there but Daniel Ellsberg and Edward Snowden did actually did also service to the country. We did not know that the NSA was tracking geolocation data of every American citizen's phone calls. That was scandalous.”

The US currently holds a lead in large language model development that China has not won, representing a unique window of opportunity to set international AI standards before competitive pressure forces either abandonment or weaponization of restraint.

“So, this is a great moment for the US to start setting international standards in the same way that the US was able to set international standards on atomic weapons when it had that lead over the Soviet Union. If we wait too long, it'll be too late.”

A segment of House Republicans skeptical of Ukraine aid is seeking material to undermine support for the war, and Russia has strategic incentive to provide that material through information warfare.

“The problem is and it will become more of a problem as the months pass that a segment of the House Republican Caucus is skeptical about the war in Ukraine and growing more skeptical with every passing week and they are looking for material to work with. And you can be sure that the Russian government has a strong incentive to provide material for them to work with because from an information warfare point of view Moscow wants to undermine Western support for Ukraine full stop.”

If the U.S. goes back to information stovepiping (compartmentalized classification), it will reduce intelligence effectiveness even if it improves security.

“I'm worried by HR saying we realized this problem and now we're going to go back to siloing, um, in information.”

The Jack Teixeira leak did not reveal world-shattering secrets; most allied governments already knew about US intelligence gathering, and Ukraine casualty figures aligned with open-source estimates, with the main embarrassment being the revelation of NATO special forces training operations in Ukraine.

“I don't think there are many European governments who are shocked shocked that the United States, uh, is, uh, is spying on them. I mean, that's just not new news. Nor was anything that came out about the war in Ukraine, uh, for example, casualties on the two sides a revelation to me. It pretty much aligned with what we had already figured out from from open-source, uh, information. I asked a a senior military, uh, figure, not HR, but someone else, if there was anything damaging that had come out. And he said the damaging thing that the new and somewhat embarrassing thing is the extent to which special forces operators from NATO countries are in Ukraine engaged in training.”

Americans spend $30 billion annually on recreational marijuana but only $18 billion on craft beer and chocolate combined, suggesting substantial demand for legal cannabis products.

“Americans spend $30 billion a year on recreational marijuana. They spend $18 billion a year on craft beer and chocolate combined.”

The current regulatory state, led by the same people who run the FDA, CDC, and Federal Reserve, is unfit to judiciously regulate AI and would likely use regulation as an instrument of political censorship.

“Who is the we? The we is our current regulatory state. And do you think that the capacity of our current regulatory state is that the same people run the FDA, the CDC, and the Federal Reserve are are going to be able to judiciously pause AI in just the right way and not turn it into a a massive attempt at political censorship?”

The debates around AI regulation replicate fundamental disagreements between a security-focused Burkean conservative position (go slow, anticipate second-order effects) and a libertarian progress-focused position (don't regulate nascent technologies).

“I think we have uh three people on this broadcast who've already passed that milestone who are giving you plenty advice on what to do...I find it to be a fascinating topic but also because I think that John and Niall do not necessarily see eye to eye on all things AI. That means H.R. that you get to play the role of peacemaker or troublemaker depending on how much trouble you want to foment here.”