User:Andnan/Draft

PauseAI

Formation	April 2023 (2023-04)
Founder	Joep Meindertsma
Founded at	Utrecht, Netherlands
Type	Advocacy group, Nonprofit
Purpose	Mitigating the existential risk from artificial general intelligence and other risks of advanced artificial intelligence
Region	International
Website	pauseai.info

PauseAI is a global political movement founded in the Netherlands with the stated aim of achieving global coordination to stop the development of artificial intelligence systems more powerful than GPT-4, at least until it is known how to build them safely. The movement was established in Utrecht in April 2023 by tech CEO Joep Meindertsma.

Citing the control problem and the alignment problem, PauseAI is concerned about the existential risk from artificial general intelligence.

Its first public action was to protest in front of Microsoft's Brussels lobbying office in May 2023 whilst they were holding an event on artificial intelligence. Later, on the 20th of October 2023, they met in Parliament Square Gardens to call for a "global moratorium" before the AI Safety Summit at Bletchley Park the following month.

Background

In March 2023, the Future of Life Institute released a letter calling for "AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4", and it has since received over 30,000 signatures. Signatories included AI researchers such as Yoshua Bengio and Stuart Russell, along with tech entrepreneurs like Elon Musk and Jaan Tallinn. The lead researcher at the Machine Intelligence Research Institute, Eliezer Yudkowsky, did not sign the letter, stating "I refrained from signing because I think the letter is understating the seriousness of the situation and asking for too little to solve it."

Proposal

PauseAI's stated goal is to "implement a pause on the training of AI systems more powerful than GPT-4". Their website lists some proposed steps to achieve this goal:

• Set up an international AI safety agency, similar to the IAEA. This agency will be responsible for:
  • Granting approval for deployments. This will include red-teaming / model evaluations.
  • Granting approval for new training runs of AI models above a certain size (e.g. 1 billion parameters).
  • Periodic meetings to discuss the progress of AI safety research.
• Only allow training of general AI systems more powerful than GPT-4 if their safety can be guaranteed.
  • By more powerful than GPT-4, we mean all AI models that are either 1) larger than 10^12 parameters, 2) have more than 10^25 FLOPs used for training or 3) capabilities that are expected to exceed GPT-4.
  • Note that this does not target narrow AI systems, like image recognition used for diagnosing cancer.
  • Require oversight during training runs.
  • Safety can be guaranteed if there is strong scientific consensus and proof that the alignment problem has been solved. Right now, this is not the case, so right now we should not allow training of such systems.
  • It may be possible that the AI alignment problem is never solved - it may be unsolvable. In that case, we should never allow training of such systems.
  • Even if we can build controllable, safe AI, only build and deploy such technology with strong democratic control. A superintelligence is too powerful to be controlled by a single company or country.
  • Track the sales of GPUs and other hardware that can be used for AI training.
• Only allow deployment of models after no dangerous capabilities are present.
  • We will need standards and independent red-teaming to determine whether a model has dangerous capabilities.
  • The list of dangerous capabilities may change over time as AI capabilities grow.
  • Note that fully relying on model evaluations is not enough.

External links

• Official website