AI Governance & Safety Canada
T1 Nonprofit Governance Active Canada
AIGS Canada is a nonpartisan nonprofit focused on AI governance and safety. Its official materials explicitly state a mission to ensure advanced AI is safe and beneficial and to catalyze Canadian leadership.
Profile
ScopeCatalyzing Canada’s leadership in AI governance and safety.
Programs / outputsAdvocacy for responsible AI governance in Canada; policy briefs on AI safety regulation; community building
PublicationsPolicy briefs at aigs.ca; advocacy materials for Canadian AI governance
PartnersCanadian AI policy ecosystem
FundingNonprofit; grant-funded
Nonprofit Mixed Active United States (verify)
Apollo Research focuses on reducing risks from dangerous capabilities in advanced AI systems, particularly scheming behaviors. It develops evaluations and conducts technical research, and it also provides governance-oriented guidance.
Profile
ScopeReducing risks from dangerous capabilities in advanced AI systems; evaluations for scheming/deception; governance guidance.
Programs / outputsModel evaluations for scheming; technical research; governance advice (per site).
Publicationshttps://www.apolloresearch.ai/research/
PartnersUK AISI (contracted deception evaluations); OpenAI (fine-tuning API red-teaming); AISF/FMF grantee; engagement with Canada AISI, EU AI Office, France, GPAI
Funding~$1.54M initial grant from Open Philanthropy (2023); oversubscribed seed round led by 50Y (2025); AI Safety Fund grant from Frontier Model Forum (Dec 2025); now a Public Benefit Corporation (PBC)
Nonprofit Training Active United Kingdom (verify)
BlueDot Impact runs cohort-based training programs on AI safety and AI governance and maintains public resources for the field. This is included as a field-building/training organization.
Profile
ScopeRuns free courses on AI safety and governance; builds community for contributors.
Programs / outputsAI safety training courses: AGI Strategy (L2), Technical AI Safety (L2), AI Governance (L2), Defensive Engineering (L2), Technical AI Safety Project Sprint (L3), AI Safety Operations Bootcamp, Biosecurity; 7,000+ alumni since 2022; Rapid Grants program for AI safety work; Career Transition Grants; expanding to 3-level defense-in-depth curriculum
Publicationshttps://bluedot.org/resources
PartnersAlumni placed at Anthropic, DeepMind, UK AISI; biosecurity grants up to £50k for graduates
Funding$35M total raised; $25M raised in 2025; all courses free (pay-what-you-want model)
Canadian AI Safety Institute (CAISI)
T1 Government Evals Active Canada
CAISI is a Government of Canada institute established to support safe and responsible AI development and deployment. Government pages and announcements provide direct evidence of its mandate.
Profile
ScopeGovernment institute supporting safe and responsible AI development/deployment in Canada.
Programs / outputsFederal AI safety institute established under Canada's AI legislation; focus on testing and evaluating advanced AI systems; collaboration with international AISI network
PublicationsEmerging; publications via ISED website; alignment with Canada's Artificial Intelligence and Data Act (AIDA)
PartnersISED (parent department); international AISI network (UK, US, EU, etc.)
FundingFederal funding through Innovation, Science and Economic Development Canada (ISED); announced as part of Canada's AI strategy
Center for AI Safety
CAIS T1 Nonprofit Mixed Active United States
The Center for AI Safety is a nonprofit explicitly focused on reducing societal-scale risks from AI. Its mission statement emphasizes safety research, field-building, and safety standards advocacy.
Profile
ScopeReducing societal-scale risks from AI via research, field-building, and advocacy.
Programs / outputsAI Safety Research (interpretability, robustness, alignment); AI And Society Fellowship program; CAIS Dashboard tracking AI incidents and metrics; field-building grants and research support
Publicationsai_memo_hierarchical_inpainting.pdf; numerous papers on interpretability, robustness, and alignment via researchers supported by CAIS grants; 2024 Impact Report
PartnersCollaborations with UC Berkeley, Stanford, MIT; AI safety research community broadly
FundingMajor funder: Jaan Tallinn; receives donations from effective altruism and longtermist communities; annual budget not publicly disclosed but substantial given scale of grant programs
Center for AI Standards and Innovation (NIST)
CAISI (U.S. rebrand context) T1 Government Standards Active United States
NIST’s CAISI is the U.S. government’s primary point of contact for AI testing, standards, and security-oriented collaboration. Reporting indicates this is the renamed successor context to the earlier U.S. AI Safety Institute framing.
Profile
ScopeTesting, evaluation, and collaborative research to harness and secure commercial AI systems.
Programs / outputsContinuation of AISI under Trump Administration AI Action Plan; focus on standards and innovation rather than safety; AI evaluation science for federal procurement; pre-deployment and post-deployment model evaluation with OpenAI and Anthropic
PublicationsSame as US AISI — transitioned branding from AISI to CAISI in 2025; publications at nist.gov/caisi
PartnersOpenAI, Anthropic (security evaluations); GSA (federal procurement evaluation); UK AISI (joint evaluations)
FundingFederal funding through NIST budget; same appropriation as former AISI
Center for Human-Compatible AI (CHAI, UC Berkeley)
T1 Academic Technical Active United States
CHAI is an academic center at UC Berkeley focused on technical and conceptual work to push AI toward provably beneficial outcomes. Its official pages explicitly state this safety-relevant mission.
Profile
ScopeReorient AI research toward provably beneficial systems (mission).
Programs / outputsResearch in: provably beneficial AI, inverse reinforcement learning, cooperative AI, human-robot cooperation, value alignment, RLHF limitations; annual CHAI Workshop (10th annual June 2026); Provably Safe and Beneficial AI (PSBAI) NSF-funded initiative; political neutrality evaluations for AI (new)
PublicationsHundreds of publications since 2016; 32+ student papers in most recent year; key papers: Open Problems of RLHF (Casper et al. 2023), STARC framework, adversarial Go policies, Tensor Trust; published at NeurIPS, ICML, ICLR, AAAI, IJCAI, ICRA, CoRL
PartnersUC Berkeley BAIR Lab; OECD (co-chair AI Futures Expert Group); UK 10 Downing Street; US Senate; UNESCO; GPAI; World Economic Forum (co-chair Global Futures Council on AI)
FundingNSF-funded (PSBAI initiative); Stuart Russell's endowed position; BAIR Lab resources; grants from OSTP, Open Philanthropy, and others
Center for Human-Compatible AI (UC Berkeley)
T1 Academic Technical Active United States
Added as part of the initial AI safety ecosystem sweep. This entry will be tightened and upgraded/dropped based on explicit mission statements and programs in later verification passes.
Profile
Programs / outputsResearch in: provably beneficial AI, inverse reinforcement learning, cooperative AI, human-robot cooperation, value alignment, RLHF limitations; annual CHAI Workshop (10th annual June 2026); Provably Safe and Beneficial AI (PSBAI) NSF-funded initiative; political neutrality evaluations for AI (new)
PublicationsHundreds of publications since 2016; 32+ student papers in most recent year; key papers: Open Problems of RLHF (Casper et al. 2023), STARC framework, adversarial Go policies, Tensor Trust; published at NeurIPS, ICML, ICLR, AAAI, IJCAI, ICRA, CoRL
PartnersUC Berkeley BAIR Lab; OECD (co-chair AI Futures Expert Group); UK 10 Downing Street; US Senate; UNESCO; GPAI; World Economic Forum (co-chair Global Futures Council on AI)
FundingNSF-funded (PSBAI initiative); Stuart Russell's endowed position; BAIR Lab resources; grants from OSTP, Open Philanthropy, and others
Centre for the Study of Existential Risk (CSER)
T1 Academic Mixed Active United Kingdom
CSER is a Cambridge research center studying existential risks, including technical and governance questions related to AI safety. Its official pages explicitly describe research on AI risks and broader catastrophic-risk mitigation.
Profile
ScopeResearch on existential and global catastrophic risks, including risks from artificial intelligence (technical + governance).
Programs / outputsInterdisciplinary research on existential and global catastrophic risks; focus areas: AI risk, biosecurity, climate, nuclear; policy engagement with UK and international governments
Publicationshttps://www.cser.ac.uk/work/
PartnersUniversity of Cambridge; collaborations with other Cambridge institutes (Leverhulme CFI, etc.); UK government advisory role
FundingUniversity of Cambridge research center; funded by university grants and philanthropic donations
FAR.AI (Frontier Alignment Research)
T1 Nonprofit Mixed Active United States (verify)
FAR.AI is a research and education nonprofit dedicated to ensuring advanced AI is safe and beneficial. It runs field-building events and supports technical progress through collaborative programs.
Profile
ScopeAI safety research & education nonprofit focused on safe and beneficial frontier AI.
Programs / outputsWorkshops, events, research incubator/acceleration; publications and updates.
Publicationshttps://far.ai/news
PartnersOpenAI (GPT-5 red-teaming); UK AISI; grantmaking to researchers at ETH Zurich, UC Santa Barbara, UC Berkeley, UMD
Funding$30M+ multi-funder support secured (2025); principal funders: Coefficient Giving, Schmidt Sciences, Survival and Flourishing Fund, CSET, AI Safety Fund (FMF), UK AISI; seeking up to $4M/year additional
Nonprofit Standards Active United States/International (verify)
The Frontier Model Forum is an industry-supported nonprofit explicitly focused on addressing significant public safety and national security risks from frontier AI models. It publishes safety evaluation best-practice briefs and supports standards and information sharing.
Profile
ScopeIndustry-supported nonprofit addressing significant risks to public safety and national security from frontier models.
Programs / outputsAI Safety Fund ($5M+ disbursed across 11 grantees Dec 2025); safety commitments and best practices for frontier AI companies; synthetic content transparency; red-teaming guidelines
Publicationshttps://www.frontiermodelforum.org/updates/
PartnersFounding members: Anthropic, Google, Microsoft, OpenAI; AISF grantees include Apollo Research, FAR.AI, and 9 others
FundingFunded by member companies (Anthropic, Google, Microsoft, OpenAI); AI Safety Fund partners: Patrick J. McGovern Foundation, David & Lucile Packard Foundation, Schmidt Sciences, Jaan Tallinn
Future of Life Institute
T1 Nonprofit Mixed Active United States
Added as part of the initial AI safety ecosystem sweep. This entry will be tightened and upgraded/dropped based on explicit mission statements and programs in later verification passes.
Profile
Programs / outputsVitalik Buterin PhD Fellowships in AI Existential Safety ($40K/yr + tuition); Vitalik Buterin Postdoctoral Fellowships ($80K/yr); US-China AI Governance PhD Fellowships; AI Existential Safety Community membership (travel support); RFPs for religious and multistakeholder AI safety projects
PublicationsFLI open letters on autonomous weapons and AI risk; policy briefs; annual reports at futureoflife.org/about-us/funding/
PartnersBeneficial AI Foundation (BAIF) partnership for postdoctoral fellowships; fellowship alumni at Stanford, UC Berkeley, Oxford, Cambridge, MIT, CMU, ETH Zurich
Funding~$17M total expenditure (2024); 49% grants to other orgs; primary funder: Vitalik Buterin endowment (2021); does not accept Big Tech or AGI-company donations; only $85K from individual/new donors in 2024
GDM Safety (Google DeepMind)
T1 Corporate Technical Active United Kingdom
Google DeepMind's safety division, conducting frontier AI safety research including alignment, evaluations, and responsible development practices for Gemini and other frontier models.
Profile
Programs / outputsFrontier safety research, alignment, responsible development, model evaluations, Gemini safety
Global Catastrophic Risk Institute
T1 Nonprofit Governance Active United States
GCRI is a nonprofit think tank focused on global catastrophic risks, including AI. It explicitly publishes AI risk governance work aimed at practical mitigation of catastrophic AI risk.
Profile
ScopeAI risk governance research as part of global catastrophic risks analysis.
Programs / outputsResearch on global catastrophic risks including AI risk, pandemics, nuclear war, climate; scenario analysis and risk assessment methodology
PublicationsPublications at gcri.org; focused on risk analysis methodology and interdisciplinary catastrophic risk assessment
PartnersCollaboration with broader existential risk community; academic partnerships
FundingSmall nonprofit; funded by grants and individual donations
GovAI (Centre for the Governance of AI)
T1 Research Governance Active United Kingdom
GovAI is a governance-focused research organization producing work and training talent to help decision-makers manage advanced AI risks. Its official pages and research listings provide direct evidence of mission and activity.
Profile
ScopeGovernance research and talent development for managing risks/opportunities from advanced AI.
Programs / outputsResearch across: AI Regulation, Technical AI Governance, AI Progress/Forecasting, Economics, Security, Law & Policy, Political Science, Survey Research; Fellowship programs (Winter, Summer, DC Fall); GovAI Policy Program (GAPP) for graduate students and professionals
Publicationshttps://www.governance.ai/research
PartnersBased at Oxford; policy engagement with UK government, EU, OECD, UNESCO, GPAI; compute governance collaboration with CSET
FundingNonprofit research center; funded by grants from EA/longtermist community and policy organizations
International AI Safety Report
T1 Coalition Mixed Active International
The International AI Safety Report is an international expert collaboration producing scientific syntheses of risks and mitigations for general-purpose AI. Official pages describe the scope and publication cycles.
Profile
ScopeScientific synthesis of risks and mitigations for general-purpose AI.
Programs / outputsAnnual international scientific assessment of AI safety; modeled on IPCC; first full report expected 2025-2026; expert consensus-building across nations
Publicationshttps://internationalaisafetyreport.org/publications
Partners30+ countries; UK AISI (coordinating); academic advisory board
FundingGovernment funding from participating nations
Leverhulme Centre for the Future of Intelligence (CFI)
T1 Academic Governance Active United Kingdom
The Leverhulme Centre for the Future of Intelligence is an interdisciplinary research center at Cambridge focused on the long-term future of intelligence, including societal impacts and governance of AI. It is included as a major safety-adjacent research institution.
Profile
ScopeInterdisciplinary research on the future of intelligence and responsible AI development/governance.
Programs / outputsInterdisciplinary research on AI impacts, governance, and societal implications; AI: Narrative and Representation project; AI and democracy research; long-term impacts of AI on human society
PublicationsAcademic publications across AI ethics, governance, and social impact; based at University of Cambridge
PartnersUniversity of Cambridge; partnership with CSER; UK AI governance ecosystem
FundingLeverhulme Trust-funded (£10M initial grant); University of Cambridge support
Machine Intelligence Research Institute
MIRI T1 Nonprofit Technical Active United States
MIRI is a long-running nonprofit focused on technical AI alignment and control research. Its official pages explicitly describe work aimed at ensuring advanced autonomous AI systems are safe and beneficial.
Profile
ScopeTechnical research on alignment/control of advanced autonomous AI systems.
Programs / outputsAlignment research; mathematical theory for trustworthy reasoning.
Publicationshttps://intelligence.org/our-research/
PartnersOpen Philanthropy (major funder); LessWrong (community platform); MIRI workshops and research retreats
FundingPrimarily funded by individual donations; Open Philanthropy major grant ($1.25M 2022, $500K 2021, $500K 2020); total annual revenue ~$1-2M
MATS (ML Alignment & Theory Scholars)
T1 Program Training Active United States
MATS is a research training program explicitly focused on advancing model safety research (control, interpretability, oversight, evaluations, red teaming). Its own materials clearly position it as an AI safety field-building pipeline.
Profile
ScopeResearch training program in model safety: control, interpretability, oversight, evals/red teaming, robustness.
Programs / outputs527+ researchers trained since 2021; 180+ research papers published (h-index 47); 5 research tracks: Empirical, Theory, Policy & Strategy, Technical Governance, Compute Infrastructure; Summer 2026 cohort: 120 fellows, 100 mentors — largest ever
Publications180+ papers with 10,000+ collective citations; publications via mentored research streams at NeurIPS, ICML, ICLR, and Alignment Forum
PartnersPartner research orgs: Anthropic Alignment Science, UK AISI, Redwood Research, ARC, LawZero; mentor streams from Google DeepMind, Epoch AI, and others
Funding$15,000 stipend + $12,000 compute per scholar; extension pathway with 6-12 months continued funding; total program budget growing substantially; funded by EA and longtermist communities
METR (Model Evaluation & Threat Research)
T1 Nonprofit Evals Active United States
METR is a research nonprofit focused on evaluating frontier AI models to understand high-stakes capabilities and risks. Its About page and public research outputs provide direct evidence of its safety-evaluation mandate.
Profile
ScopeIndependent evaluation of frontier models for catastrophic-risk-relevant capabilities.
Programs / outputsFrontier model evaluations; datasets on eval integrity threats (examples on research page).
Publicationshttps://metr.org/research/
PartnersAnthropic, OpenAI (pre-deployment evaluation partnerships); publishes evaluations independently for open-weight models
FundingNot-for-profit; does not accept compensation for evaluations; funded by grants and philanthropic support
MIT AI Alignment (MAIA)
MAIA T1 Program Training Active United States
MAIA is a MIT student group explicitly conducting research aimed at reducing risks from advanced AI. It functions as a training/field-building org with a clear safety mission.
Profile
ScopeStudent-led research group reducing risk from advanced AI.
Programs / outputsResearch in mechanistic interpretability, model organisms of deception, alignment tax, scalable oversight; part of MIT CSAIL
PublicationsAcademic publications at top ML venues via MIT CSAIL
PartnersMIT CSAIL; collaborations with broader alignment research community
FundingMIT-funded; grants from Open Philanthropy and other alignment funders
OECD.AI (OECD AI Policy Observatory)
T1 IGO Governance Active France (OECD HQ)
OECD.AI is an intergovernmental policy observatory supporting trustworthy AI via principles, policy tracking, and publications. It is included as a global governance infrastructure node.
Profile
ScopeTrustworthy AI principles and global policy tracking and guidance.
Programs / outputsOECD AI Policy Observatory (live database of 1000+ AI policy initiatives across 80+ countries); AI Principles (adopted 2019, updated 2024); AI incident tracking; Global AI Expert Network; work on AI risk classification, compute governance, and international coordination
PublicationsOECD AI Principles (2019, updated 2024); G7 Hiroshima Process on Generative AI; regular policy observatory reports and dashboards; numerous policy briefs and working papers
Partners38 OECD member countries; G7; Global Partnership on AI (GPAI); UNESCO; International AI Safety Network
FundingOECD member country contributions; intergovernmental organization budget
Corporate Mixed Active United States
OpenAI's safety division, responsible for the preparedness framework, red-teaming, and safety evaluations for GPT and o-series models.
Profile
Programs / outputsFrontier model safety, preparedness framework, red-teaming, model safety evaluations, o-series safety
Nonprofit Mixed Active United States
Redwood Research is a nonprofit AI safety and security research organization focused on threat assessment and mitigation for AI systems. Its public research pages cover applied alignment/control and evaluations-related work.
Profile
ScopeThreat assessment/mitigation for AI systems; applied alignment/control; evals.
Programs / outputsAI control; evaluations; alignment faking case study (examples on research pages).
Publicationshttps://www.redwoodresearch.org/research
PartnersCollaborations with UC Berkeley, Stanford, and other alignment labs
FundingBacked by Nat Friedman and Daniel Gross; non-profit research lab; specific funding amounts not publicly disclosed
Nonprofit Mixed Active France
SaferAI is a France-based nonprofit working on AI risk management through research, policy, standards, and risk measurement tools (including company risk-management ratings). Its official pages clearly state an AI safety mission.
Profile
ScopeAI risk measurement, risk management ratings, standards and policy work to make AI safer.
Programs / outputsAI model safety ratings and evaluations; risk management assessment framework for frontier AI companies
Publicationshttps://ratings.safer-ai.org/
PartnersAI companies evaluated (confidential); policy community engagement
FundingNonprofit; grant-funded
U.S. AI Safety Institute (NIST)
U.S. AISI T1 Government Standards Active United States
The U.S. AI Safety Institute (housed within NIST) publishes guidance and strategic materials aimed at mitigating risks from advanced AI. Official documents explicitly describe the institute’s safety mandate.
Profile
ScopeRisk mitigation guidance and safety mechanisms for advanced AI models/systems (as stated by NIST).
Programs / outputsAI Safety Institute Consortium (AISIC) with 5 working groups: Risk Management for Generative AI, Synthetic Content, Capability Evaluations, Red-Teaming, Safety & Security; Practices for Automated Benchmark Evaluations of Language Models (IPD Feb 2026); MOU with GSA for federal AI procurement evaluation
PublicationsPractices for Automated Benchmark Evaluations of Language Models (IPD, Feb 2026); AI RMF companion for generative AI; NIST AI 100-1 through 100-5 series
PartnersMOU agreements with OpenAI and Anthropic (pre-deployment model access); partnership with UK AISI; GSA partnership for federal AI procurement evaluation; AISIC consortium of 200+ organizations
FundingFederal funding through NIST budget; AISI Consortium membership fees not publicly disclosed; part of CHIPS and Science Act AI provisions
UK AI Security Institute
UK AISI T1 Government Evals Active United Kingdom
The UK AI Security Institute is a government body focused on evaluating advanced AI capabilities and mitigations. Its official mission aligns directly with safety evaluation and risk reduction work.
Profile
ScopeUnderstanding capabilities/impacts of advanced AI and testing risk mitigations.
Programs / outputs6 risk research domains: Cyber Misuse, Criminal Misuse, Autonomous Systems, Dual-Use Science, Societal Resilience, Human Influence; 5 solutions teams: Safeguard Analysis, Control, Alignment, Science of Evaluations, Capabilities Post-Training; Inspect evaluation framework; ControlArena; tested 30+ frontier AI models; Frontier AI Trends Report 2025
Publications30+ publications in 2025-2026 including: AISI Frontier AI Trends Report (Dec 2025); Science paper on AI persuasion (76K+ participants); RepliBench; boundary point jailbreaking; control monitoring; sandbagging evaluations; published at aisi.gov.uk/research
PartnersMOU partnerships with OpenAI, Anthropic, Google DeepMind, Cohere; International Network for Advanced AI Measurement (INAIM); GSA (US); NIST/CAISI (US)
FundingThe Alignment Project: £15m; Systemic Safety Grants: £8m; Challenge Fund: £5m; government-backed with 100+ technical staff; UK DSIT funding
Program Training Active
AI Safety Camp is an online part-time program that teams participants to work on concrete AI safety research projects. Its site publishes cohorts, projects, and research outputs.
Profile
ScopeOnline, part-time AI safety research program organizing project teams.
Programs / outputsBiannual camps connecting researchers to AI safety projects and mentors; project-based learning format; career pathway into alignment research
Publicationshttps://www.aisafety.camp/research-outputs
PartnersAlignment research community; connects to MATS, SERI, and other training programs
FundingNonprofit; funded by EA/longtermist community grants
For-profit Technical Active United Kingdom
Conjecture is an alignment-focused startup that explicitly frames its work around the controllable, safe development of advanced AI. Its site publishes alignment-focused essays and research updates.
Profile
ScopeAlignment research startup; building controllable, safe development of advanced AI.
Programs / outputsAlignment research program; public essays on alignment strategy.
Publicationshttps://www.conjecture.dev/research
PartnersFiscal sponsorship of SERI MATS London cohort and ARENA; London alignment ecosystem building
FundingVC-backed by Nat Friedman, Daniel Gross, Patrick & John Collison, Andrej Karpathy, Arthur Breitman; founders retain complete control; revenue-generating via Lemma Labs products
IP
International Programme on AI Evaluation (ai-evaluation.org)
T1 Program Evals Active Spain (Valencia; program location)
The International Programme on AI Evaluation is an academic program focused on evaluating AI capabilities and safety, with a defined 2026 schedule. It is included as an evaluations-focused training initiative.
Profile
ScopeAcademic program dedicated to AI evaluation focusing on capabilities and safety.
Programs / outputsInternational coordination on AI evaluation standards and methodology; bridging technical and policy communities; developing evaluation frameworks
PublicationsPolicy briefs and evaluation methodology reports at ai-evaluation.org
PartnersOECD; UK AISI; international AI safety network
FundingGovernment and foundation support
SS
Safe Superintelligence Inc.
SSI T1 For-profit Technical Active United States
Safe Superintelligence Inc. explicitly frames its entire mission and product roadmap around building 'safe superintelligence.' Its official site states a single-goal focus, and independent references corroborate the company’s existence and framing.
Profile
ScopeBuilding 'safe superintelligence' as sole product/mission.
Programs / outputsStraight-shot SSI lab (stated mission).
PublicationsNone publicly released; Ilya Sutskever has stated they will share research when ready
PartnersNo partnerships announced; operates as standalone research lab
Funding$1B raised (Sep 2024); reportedly raising $1B+ at $30B valuation (Feb 2025); co-led by Ilya Sutskever after departing OpenAI; investors include Nat Friedman, Daniel Gross, Patrick & John Collison, Andrej Karpathy; CEO Daniel Gross departed Jul 2025 for Meta
Ada Lovelace Institute
T2 Nonprofit Governance Active United Kingdom
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeAI ethics & governance org.
Programs / outputsFour thematic areas: Emerging Technology & Industry Practice; Law & Policy; Public Participation & Research; Social and Economic Policy
BRAID Programme (co-launched with University of Edinburgh and BBC, funded by AHRC) for responsible AI research
Citizens' Biometrics Council informing ICO guidance on biometric technologies
Over 40 research projects covering AI accountability, public participation, and frontier AI safety
PublicationsNavigating the Future: AI in career guidance for young people (Apr 2026)
Risky Business: AI liability analysis in the UK (Dec 2025)
Great (public) expectations: Public polling on AI governance (Dec 2025)
Over half of Ada's 18 recommendations implemented in EU AI Act
PartnersAlan Turing Institute; Royal Society; British Academy; Royal Statistical Society; Nuffield Council on Bioethics; Wellcome Trust; techUK; Luminate; University of Edinburgh; BBC; Arts and Humanities Research Council
FundingFounded and primarily funded by the Nuffield Foundation; independent of government and tech industry; receives partnership funding from AHRC for BRAID programme
Ada Lovelace Institute (AI ethics & governance)
T2 governance Ensuring AI and data work for people and society; addressing ethical, social, and legal risks including algorithmic accountability and biometrics governance active UK
Ada Lovelace Institute (AI ethics & governance) is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeIncluded as part of the AI safety ecosystem; mission verification may be needed for safety-first criteria.
Programs / outputsPolicy-facing research on AI regulation; reports on algorithmic accountability and facial recognition governance; advocates for public participation in technology oversight; leading European bridge between technical research and human-rights policy
FundingEstablished by Nuffield Foundation in 2018; total funding approximately USD 5 million
AI Incident Database (AIID)
T2 Resource Evals Active United States
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeIncident tracking; evaluation data.
FundingPatrick J. McGovern Foundation; donations; Responsible AI Collaborative nonprofit
AI Incident Database (Partnership on AI / AIID)
T2 Resource Evals Active United States
AI Incident Database (Partnership on AI / AIID) is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeIncluded as part of the AI safety ecosystem; mission verification may be needed for safety-first criteria.
FundingPatrick J. McGovern Foundation; donations; Responsible AI Collaborative nonprofit
Nonprofit Governance Active United States
AI Now Institute is a policy research organization focused on accountability and redirecting AI development trajectories toward public interest outcomes. It is included as part of the safety governance ecosystem.
Profile
ScopePolicy research challenging current AI trajectory; accountability and societal risk governance.
Programs / outputsAnnual Landscape Report mapping AI market dynamics, industry power, and policy strategies (2025: Artificial Power)
Research across 10 focus areas: accountability, biometrics, climate, geopolitics, inequality, labor, markets, privacy, public interest AI, safety & security
Policy advocacy including testimony at Philadelphia City Council, remarks before the UN General Assembly on AI Governance
North Star Data Center Policy Toolkit for state/local intervention against AI data center expansion
PublicationsArtificial Power: 2025 Landscape Report (Jun 2025)
North Star Data Center Policy Toolkit (Dec 2025)
Fission for Algorithms: The Undermining of Nuclear Regulation in Service of AI (Nov 2025)
Report on National Security Risks from Weakened AI Safety Frameworks (Apr 2025)
PartnersIndependent from NYU since 2022; no corporate funders; partners with civil society organizations for policy advocacy
FundingPackard Foundation $600,000 grant (2026, 36-month term); does not accept corporate/funding from tech companies; funded exclusively by foundations
Nonprofit Governance Active United States
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeAI policy research and advocacy.
FundingAnonymous tech/finance donors; foundation grants; revenue from research contracts
AI Policy Institute (AIPI)
T2 Policy Research
The AI Policy Institute (AIPI) is a US-based nonpartisan policy research organization focused on the governance of artificial intelligence. It monitors AI industry lobbying, publishes polling data on public attitudes toward AI regulation, and advocates for legislative oversight of frontier AI systems.
Profile
ScopeUS AI policy research and advocacy; focuses on AI governance, election integrity, and corporate accountability. Tracks AI industry lobbying and policy positions.
Programs / outputsPublic Opinion Tracking: regular YouGov polls of US voters on AI attitudes and policy preferences
Policy Research: reports on AI threats, regulatory gaps, and policy interventions for catastrophic risk
Media and Policymaker Engagement: bridging AI community with journalists and lawmakers; met with 24+ lawmakers as of late 2023
Coalition for Responsible AI (campaign during 2025 federal election)
PublicationsInaugural poll: 83% of voters believe AI could accidentally cause a catastrophic event; 82% prefer slowing AI development
Poll: Voters Want Rules on Deep Fakes, International Standards, and Other AI Safeguards
Policy research on regulatory gaps and catastrophic AI risk
PartnersYouGov (polling partner); AI Policy Network/AIPN (affiliated 501(c)(4) advocacy arm with shared executive director Daniel Colson); coverage in Axios, Vox, Semafor
FundingAnonymous tech/finance donors; foundation grants
AI Risk and Vulnerability Alliance (ARVA)
T2 Technical Safety / Vulnerability Research
The AI Risk and Vulnerability Alliance (ARVA) maintains the AVID (AI Vulnerability Database), an open-source structured taxonomy of AI vulnerabilities including biases, security failures, and performance gaps. AVID enables organizations to document and search AI failures in a standardized format analogous to CVE in cybersecurity.
Profile
ScopeMaintains AVID (AI Vulnerability Database) — a structured taxonomy and database of AI failure modes, biases, and vulnerabilities across models and datasets.
Programs / outputsAVID (AI Vulnerability Database): open-source knowledge base of failure modes for GPAI systems including open-weight models, closed-API systems, and AI agents
ARVA Response to NTIA AI Accountability Policy Request for Comment
Support for open letter on voluntary safe harbor protections for good faith testing of generative AI systems
Community-driven vulnerability reporting and model evaluation tools
PublicationsAVID: AI Vulnerability Database rebuilt for the age of AI agents
ARVA Response to NTIA AI Accountability Policy Request for Comment
ARVA support for open letter on safe harbor protections for good faith AI testing
PartnersBoard includes researchers from Microsoft (Kush Varshney), Accenture (Subho Majumdar), and independent AI ethics practitioners (Rumman Chowdhury)
AI Safety Connect (AISC)
T2 Convening / Diplomacy Global AI governance coordination; AI red lines campaign International
Invitation-only convening initiative founded 2025 by Cyrus Hodes and Nicolas Miailhe; launched at the Paris AI Action Summit (Feb 2025) and convened ~100 high-level participants at UNGA 2025, framing AI safety as a diplomatic coordination challenge.
Profile
Programs / outputsGlobal Call for AI Red Lines (UNGA 2025); high-level invitation-only convenings at Paris AI Action Summit and UNGA
field-building Mitigating catastrophic AI risks by guiding individuals into the AI safety ecosystem and matching them with relevant projects and communities active
AI Safety Quest is included as an AI safety ecosystem node. Community that helps people navigate the AI safety ecosystem and find projects. This row is intended for coverage/auditability and may be excluded in a stricter 'orgs only' canonicalization.
Profile
ScopeCommunity that helps people navigate the AI safety ecosystem and find projects.
Programs / outputsFree 1-on-1 career navigation calls; mentorship for emerging talent; community-building Quest Parties; 400+ advisees served since 2023
FundingFully volunteer-based; no external funding reported
AI Safety Support (AISafety.training)
T2 resource-hub Field-building and improving knowledge accessibility for the AI safety community active
Added as part of the initial AI safety ecosystem sweep. This entry will be tightened and upgraded/dropped based on explicit mission statements and programs in later verification passes.
Profile
ScopeOperates as infrastructure and resource aggregation layer for the field — a centralised database rather than a direct training provider. Distinct from peer training orgs by focusing on discoverability and coordination.
Programs / outputsLots of Links — extensive directory and resource compilation for AI safety professionals
AI Watch (European Commission JRC)
T2 Government Governance Active Belgium
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeEU monitoring and policy support for AI.
Programs / outputsAI Watch Index: 28 indicators across 6 policy-relevant dimensions for assessing EU AI performance
AI in Europe Map: country-level AI strategy, landscape, and investment data dashboard
Monitoring of AI industrial, technological, and research capacity across EU Member States
Generative AI Outlook Report and AI Skills Supply and Demand analysis
PublicationsThe Role of Artificial Intelligence in Scientific Research (Oct 2025)
AI Skills Supply and Demand (Oct 2025)
Generative AI Outlook Report (Jun 2025)
National Strategies on AI: A European Perspective, 2022 Edition
PartnersGerman AI Observatory; AI 4 Belgium; OECD AI Policy Observatory; Stanford AI Index
FundingEuropean Commission Joint Research Centre (JRC) funded program; originally operational 2018-2022, now broadened in scope
Nonprofit Governance Active Canada
This organization appears on multiple curated AI safety maps. It will be upgraded once primary-source mission statements and concrete programs are captured.
Profile
Programs / outputsAnnual Plan for Canada white paper series (2023, 2024, 2025) with policy recommendations adopted by government
Parliamentary committee testimonies (House of Commons Science & Research, Industry & Technology, Ethics, Canadian Heritage; Senate Social Affairs, Transport and Communications)
Testified alongside Geoffrey Hinton and David Duvenaud at Senate; INDU committee testimony reached 1 million views on Instagram
AI & Data Act (Bill C-27) brief with recommended amendments; Compute Access Fund submission on AI safety implications
PublicationsPreparing for the AI Crisis: A Plan for Canada (2025 white paper)
Governing AI: A Plan for Canada (2024 white paper)
Governing AI: A Plan for Canada (2023 white paper, original)
Submissions on AI & Data Act Bill C-27, Compute Access Fund, Directive on Automated Decision-Making
PartnersGovernment of Canada / ISED; Geoffrey Hinton and David Duvenaud (Senate testimony partners)
FundingGovernment of Canada (ISED); private donors
AISafety.com (hub/resources)
T2 Resource Field-building Active
AISafety.com is a resource hub for AI existential safety, hosting directories, resources, and ecosystem tools. It is included as a field-building infrastructure node.
Profile
ScopeResource hub supporting AI existential safety ecosystem.
AISafety.com Reading Group
T2 Resource Field-building Active
AISafety.com Reading Group is included as an AI safety ecosystem node. Fortnightly meetings discussing AI safety papers and essays (community). This row is intended for coverage/auditability and may be excluded in a stricter 'orgs only' canonicalization.
Profile
ScopeFortnightly meetings discussing AI safety papers and essays (community).
Algorithmic Justice League
T2 Nonprofit Governance Active United States
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeFairness/harms; safety-adjacent.
Algorithmic Justice League (AJL)
T2 AI Fairness / Advocacy
The Algorithmic Justice League (AJL), founded by Joy Buolamwini, researches and advocates against algorithmic bias. AJL is best known for the Gender Shades study demonstrating racial and gender bias in commercial facial recognition systems, and continues to document AI harms to underrepresented communities through research, art, and policy advocacy.
Profile
ScopeResearch and advocacy on algorithmic bias, with focus on facial recognition, hiring systems, and AI harms to marginalized communities.
Programs / outputsGender Shades Project: systematic investigation of intersectional accuracy disparities in commercial facial recognition (IBM, Microsoft, Face++)
#FreedomFlyers Campaign: investigating TSA facial recognition across 250+ US airports; produced Comply To Fly report
CRASH Project (Community Reporting of Algorithmic System Harms): bug-bounty-style platform for reporting algorithmic bias
Coded Bias documentary (Sundance 2020, Netflix, PBS Independent Lens, Emmy-nominated)
PublicationsGender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification (Buolamwini & Gebru, FAT* 2018, 4900+ citations)
Actionable Auditing: Investigating the Impact of Biased Performance Results (Buolamwini & Raji)
Change From the Outside: policy interventions for independent external AI audits (Jun 2023)
Comply To Fly?: traveler experiences and biases in TSA biometric screening
PartnersMIT Media Lab (Gender Shades collaboration with Timnit Gebru); IBM, Microsoft, Face++ (audited in Gender Shades); influenced municipal bans on facial recognition in San Francisco, Oakland, Boston, Portland
FundingFord Foundation $78,670 (2020); MacArthur Foundation $250,000 (2023); Rockefeller Foundation $150,000 (2020); Democracy Fund $175,000 (2021); fiscally sponsored project (initially New Venture Fund, later Code for Science and Society)
All Tech Is Human (AI Safety Institutes Landscape)
T2 Nonprofit Governance Active United States (org HQ not verified here)
All Tech Is Human published a detailed report cataloguing AI Safety Institutes worldwide and analyzing their role as a governance model. This org is included for the institutional safety ecosystem rather than technical alignment R&D.
Profile
ScopePublishes a report cataloguing AI Safety Institutes worldwide; included as governance/meta-source org.
Publicationshttps://alltechishuman.org/all-tech-is-human-blog/the-global-landscape-of-ai-safety-institutes
FundingMacArthur Foundation; Schmidt Futures; various foundations
For-profit Technical Active United States
This organization appears on multiple curated AI safety maps. It will be upgraded once primary-source mission statements and concrete programs are captured.
Profile
Programs / outputsConstitutional AI (CAI/RLAIF framework); Responsible Scaling Policy v3.0 (ASL-3 since May 2025); Frontier Safety Roadmap (Apr 2026, time-bound goals); Constitutional Classifiers (jailbreak defense, 95% block rate); Claude Code agentic coding tool ($2.5B run-rate); Agent Skills framework; interpretability program (circuit tracing, sparse autoencoders); Claude's Constitution published (CC0 1.0, Jan 2026)
PublicationsConstitutional Classifiers (arXiv:2501.18837, Feb 2025); Tracing the Thoughts of a LLM + Biology of a LLM (Mar 2025); Responsible Scaling Policy v3.0 (Feb 2026); RSP Reflections essay; original Constitutional AI paper (arXiv:2212.08073)
PartnersAmazon/AWS ($100B+ over 10 years, 5GW compute, 100K+ Bedrock customers); Microsoft Azure ($30B compute commitment); NVIDIA (deep tech partnership); Salesforce (Agentforce integration, full trust boundary); Accenture (Anthropic Business Group, 30K practitioners); Snowflake ($200M partnership, 12.6K customers); 8 of Fortune 10 are Claude customers
Funding~$72.3B total raised. Series G: $30B at $380B valuation (Feb 2026). Revenue: $14B run-rate (Feb 2026), 10x annual growth for 3 consecutive years. 500+ customers spending >$1M/year. Strategic investors: Google, Amazon, Microsoft, NVIDIA, Salesforce, Cisco
Resource Field-building Active
Arb Research is included as an AI safety ecosystem node. Publishes an impact assessment of AI Safety Camp. This row is intended for coverage/auditability and may be excluded in a stricter 'orgs only' canonicalization.
Profile
ScopePublishes an impact assessment of AI Safety Camp.
Programs / outputsBoutique AI safety consulting and research. 732 AI safety forecasting questions (commissioned by Open Philanthropy); Shallow Review of AI Safety (annual, 3x expanded in 2025); AI Bias Research (published in PNAS Jul 2025); Trade book on AI via Stripe Press (Oct 2025); AI Safety Camp impact assessment; 37 projects completed in 2025. Team: 4.8 FTE.
PartnersClients include Stripe, Coefficient Giving, Schmidt Futures, Mercatus Center, FAR AI, Institute for Progress. Collaborators include Poseidon (NeurIPS review event), Renaissance Philanthropy (scientific breakthroughs collection).
FundingOpen Philanthropy/Coefficient Giving (forecasting program grants); Lightspeed Grants (open-ended grant for AI and forecasting); Emergent Ventures (small grant); Stripe Press (book deal). Fee-for-service consulting for Stripe, Mercatus Center, FAR AI, Institute for Progress, and others.
governance AI governance focused on mitigating advanced-AI and global catastrophic risks; developing pathways for impactful careers active
This organization appears on multiple curated AI safety maps. It will be upgraded once primary-source mission statements and concrete programs are captured.
Profile
ScopeSpecialises in transitioning mid-to-senior professionals from law, policy, and adjacent fields into advanced AI governance through intensive cohort-based programmes. Distinct from academic peers by its practitioner focus and senior professional intake.
Programs / outputsAI Governance Taskforce — 12-week fellowship for mid-career/senior professionals producing policy-relevant research papers and briefs; fully remote, part-time (8 hrs/week)
FundingRecommended by Open Philanthropy; AI Governance Research Fellowship grant approximately USD 540,344 (GBP 401,027)
research Neuroscience-informed approaches to AGI and AI safety; creating open-science public goods through high-agency research active USA
This organization appears on multiple curated AI safety maps. It will be upgraded once primary-source mission statements and concrete programs are captured.
Profile
ScopeCreates open public goods to prepare humanity for the fundamental reshaping brought by advanced AI. Uniquely provides massive compute infrastructure alongside residency funding, distinguishing it from grant-only philanthropies. Backed by Jed McCaleb (Stellar/Ripple co-founder) with .6B in total assets.
Programs / outputsAstera Residency Program — 12-18 month fully funded residency (salary USD 125k-250k); access to 24,000 NVIDIA HGX H100s; physical hub in Emeryville California; 2x annual application cycles
FundingSelf-funded philanthropic entity; total assets USD 2.6 billion; USD 83.3 million in total historical giving
Brookings Institution (AI Policy)
T2 Policy Research
The Brookings Institution is a major US nonpartisan policy think tank. Its AI-focused work spans governance frameworks, economic impacts, national security dimensions of AI, and international AI competition. Brookings researchers regularly testify before Congress and advise executive agencies on AI policy.
Profile
ScopeNonpartisan US think tank; AI policy research covers regulation, governance frameworks, labor impacts, national security, and global AI competition.
Programs / outputsAI Policy Idea Incubator: launched 2023, convenings on AI regulation, competition, labor markets, and productivity
Artificial Intelligence and Emerging Technology Initiative (AIET): directed by Elham Tabassi (formerly NIST), advancing good governance of transformative technologies
TechPolicy Bridge: Forum for Cooperation on AI (FCAI) with CEPS; Global Task Force on AI in Education
Agentic AI Evaluation Project: partnership with Carnegie Mellon University and UC Berkeley (2025-2026)
PublicationsArtificial Intelligence and Algorithmic Exclusion (Dec 2025)
Generative AI, the American Worker, and the Future of Work (Oct 2024)
The Bletchley Park Process Could Be a Building Block for Global Cooperation on AI Safety (Oct 2024)
For AI to Make Government Work Better, Reduce Risk and Increase Transparency (Jan 2025)
PartnersCarnegie Mellon University; UC Berkeley; CEPS (Forum for Cooperation on AI); 7 governments in FCAI (Australia, Canada, EU, Japan, Singapore, UK, US)
FundingBrookings standard disclosure: supported by diverse array of funders; specific AI program funding not itemized in search results
Brookings Institution AI policy (safety governance)
T2 Nonprofit Governance Active United States
Brookings Institution AI policy (safety governance) is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeIncluded as part of the AI safety ecosystem; mission verification may be needed for safety-first criteria.
CAISI Research Program (CIFAR)
T2 Academic Research Program
The CAISI (Canadian AI Safety Institute) Research Program, housed within CIFAR (Canadian Institute for Advanced Research), funds multi-disciplinary AI safety research in Canada. It supports work on AI robustness, trustworthiness, and societal risk, and connects Canadian researchers with international AI safety networks.
Profile
ScopeCanadian AI safety research program within CIFAR; funds projects on trustworthy AI, robustness, and societal implications of advanced AI.
Programs / outputs12 new research projects across 3 priority areas: Safeguarding Society, Building Trust & Fairness, Securing Critical Systems
CIFAR AI Safety Postdoctoral Fellows program
AI Safety Scientists & Engineers at Amii, Mila, and Vector Institute
Vector Institute frontier AI model evaluation study: 11 models against 16 benchmarks, open-sourced
PublicationsCAISI Year in Review (2025 report)
CIPHER Project: AI tool to counter Russian disinformation campaigns
Mila AI Safety Studio: guardrails and benchmarks for youth protection from AI chatbots
Safeguarding Courts from Synthetic AI Content (Solution Network)
PartnersGovernment of Canada / ISED; National Research Council Canada; Amii; Mila; Vector Institute; IDRC; UK AI Security Institute; Yoshua Bengio; Geoffrey Hinton
Funding$2.4M CAD invested in first year (2025) for 12 projects; Government of Canada/ISED as primary funder; CIFAR as administrative home
CAISI Research Program at CIFAR
T2 Program Technical Active Canada
CIFAR hosts the CAISI Research Program described as multidisciplinary research on AI safety. Included as a program-level node linked to the Canadian AI Safety Institute.
Profile
ScopeMultidisciplinary research program tackling AI safety issues.
Carnegie Endowment - AI policy
T2 policy-research Researching how AI reshapes global governance, geopolitics, democratic institutions, and AI in warfare active United States
Carnegie Endowment - AI policy is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeIncluded as part of the AI safety ecosystem; mission verification may be needed for safety-first criteria.
Programs / outputsTechnology and International Affairs Program; research on AI governance across democracies and autocracies; military AI policy; international AI cooperation frameworks; offices in Washington, Beijing, Brussels
FundingCarnegie Corporation of New York (primary endowment ~$400M); Rockefeller Foundation; MacArthur Foundation; government contracts
Center for Security and Emerging Technology (CSET)
T2 Academic Governance Active United States
CSET is included as a governance ecosystem node frequently referenced in AI policy and security contexts. This entry should be upgraded once its official mission and AI safety relevant programs are directly sourced.
Profile
ScopeAI policy, national security, and emerging tech governance; safety-adjacent.
Programs / outputsEmerging Technology Observatory (ETO) — 10 public data tools (PARAT, Scout, Map of Science, AGORA, Supply Chain Explorer, Research Almanac, Country Activity Tracker, PATHWISE, AI Chip Sales Data Explorer); AI System-to-Model Innovation research; CyberAI (70 publications); China analysis (65 publications); Compete (60 publications); Workforce (55 publications); Supply Chains (26 publications); Bio-Risk (19 publications); CSET Forum membership program; Foundational Research Grants program
PublicationsChina's Military AI Wish List (Feb 2026); AI System-to-Model Innovation (Jul 2025); Promoting AI Innovation Through Competition (May 2025); CSET Recommendations for AI Action Plan (Mar 2025); Putting Explainable AI to the Test (Feb 2025). 234 reports, 229 translations, 59 data briefs, 55 data snapshots, 36 testimonies, 18 data visualizations, 4 annual reports total.
PartnersGeorgetown University Walsh School of Foreign Service (home institution). Extensive Congressional engagement (36 testimonies). Government agencies, international organizations, and tech companies. ETO platform serves public and policy audiences.
FundingFounded 2019 with $55M grant from Open Philanthropy. Second round $42M in 2021, total >$100M through 2025. Additional donors: Craig Falls, Google.org, Musk Foundation, NobleReach Foundation, Patrick J. McGovern Foundation, William and Flora Hewlett Foundation ($1M-$9.9M tier), Apple, Chan Zuckerberg Initiative, Google Research, Leidos, Microsoft, National Science Foundation, Nvidia, Scale AI, Schmidt Sciences, Rockefeller Foundation, Smith Richardson Foundation.
Centre for Security and Emerging Technology (CSET)
T2 Academic Governance Active United States
Centre for Security and Emerging Technology (CSET) is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeIncluded as part of the AI safety ecosystem; mission verification may be needed for safety-first criteria.
FundingOpen Philanthropy ~$55M over 5 years; MacArthur Foundation; Hewlett Foundation; Georgetown University
Centre for the Governance of AI
GovAI T2 governance AI policy and governance research aimed at guiding international decision-makers on societal impacts of advanced AI active United Kingdom
GovAI is widely referenced in AI governance and safety ecosystems as a key research organization focused on governance mechanisms and policy. This entry is corroborated by governance overviews and safety landscape maps.
Profile
ScopeAI governance research for risk mitigation and policy design.
Programs / outputsPolicy research on AI governance; GovAI Fellowship — 3-month program for professionals transitioning into AI governance roles; independent since 2021 when FHI closed
FundingAI Risk Mitigation Fund (USD 231,608); Survival and Flourishing Fund; Coefficient Giving; total funding approximately USD 13 million
Nonprofit Governance Active United States
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeAI governance/harms research.
Programs / outputsFour core research tracks: Labor Futures, AI on the Ground, Trustworthy Infrastructures, Climate/Technology/Justice
Algorithmic Impact Methods Lab (AIMLab): developing methodologies for empirical, participatory algorithmic impact assessments
Public Technology Leadership Collaborative: peer learning collective of scholars, researchers, and government leaders
Policy Engagement program advancing equity and justice in technology policy
PublicationsRed-Teaming in the Public Interest (Singh, Blili-Hamelin, Anderson, et al.)
Scam GPT: GenAI and the Automation of Fraud (Swartz, Marwick, Larson)
Generative AI and Labor (Nguyen, Mateescu)
Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability (published in ACM FACC*T)
PartnersPartnership on AI; UNICEF; Notre Dame-IBM Tech Ethics Lab; Public Interest Technology Infrastructure Fund / New Venture Fund
FundingMultiple major foundation funders including Ford Foundation, Hewlett Foundation, Knight Foundation (2024-2029), MacArthur Foundation (2024-2026), MacArthur Foundation, Omidyar Network, Open Society Foundations, Robert Wood Johnson Foundation (2025-2027), Rockefeller Bros Fund, Siegel Family Endowment, NSF (2024-2026), Craig Newmark Philanthropies
Resource Field-building Active
Effective Thesis is included as an AI safety ecosystem node. Program empowering students to use theses as a pathway to impact (career support). This row is intended for coverage/auditability and may be excluded in a stricter 'orgs only' canonicalization.
Profile
ScopeProgram empowering students to use theses as a pathway to impact (career support).
Programs / outputs8-Week Accelerator program for final-year students choosing impactful thesis topics or career paths
3-Month Fellowships: working with high-impact partner organizations on meaningful research
1:1 Advising: free personalized guidance on thesis topics, research design, and career planning
AI Safety-specific coaching partnerships with AI Safety Quest and expert advisors
PartnersAI Safety Quest (advising calls); Magnify Mentoring; Mental Health Navigator; alumni placed at Anthropic, MIT, Institute for Progress
FundingPaused internal coaching services due to funding constraints; currently relies on external coaches and partner organizations; listed on Effective Altruism Opportunities Board
Nonprofit Governance Active United States
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeTracks AI progress; safety-adjacent metrics.
Programs / outputsEpoch Capabilities Index (ECI, launched Oct 2025, co-developed with DeepMind Rosetta Stone); GATE model (macroeconomic framework for AI-driven growth); AI in 2030 Report (commissioned by DeepMind); FrontierMath benchmark (with OpenAI, extending to unsolved problems with Schmidt Sciences); Data Explorers (4 new in 2025); AI Chip Sales Data Explorer (Jan 2026); Active consulting for EU AI Office, UK DSIT, Sequoia Capital, Bridgewater Associates
PartnersGoogle DeepMind (Rosetta Stone, model evaluations, AI in 2030 commission); OpenAI (FrontierMath); METR (software engineering benchmark); xAI (model evaluations); EPRI (joint energy report); EU AI Office (technical consultations); UK DSIT (consultations); Sequoia Capital; Bridgewater Associates.
Funding$10.3M raised in 2025 (+40% from 2024); $5M spent in 2025 (+70%). Major donors: Coefficient Giving ($8.5M Apr 2025, $4.13M Apr 2024, $6.92M Apr 2023); Jaan Tallinn ($600K Jan 2025); Likith Govindaiah ($400K); Leopold Aschenbrenner ($200K via Manifund); Sentinel Bio ($85K); Carl Shulman ($100K); Schmidt Sciences (undisclosed Dec 2025). Spun out to independent 501(c)(3) in early 2025.
European Commission AI Office
T2 Government Governance Active Belgium
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeEU governance office.
FundingEU general budget; AI Office established under EU AI Act (2023)
European Commission AI Office (governance)
T2 government Regulating systemic AI risks, standardizing safety evaluations, and implementing the EU AI Act across 27 European member states active Belgium/EU
European Commission AI Office (governance) is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeIncluded as part of the AI safety ecosystem; mission verification may be needed for safety-first criteria.
Programs / outputsEnforces EU AI Act; voluntary Code of Practice for GPAI models (Microsoft, OpenAI, Google, Anthropic, Mistral signatories); whistleblower tool for reporting regulatory breaches; coordinating cross-EU AI policy
FundingEuropean Commission/European Union government body; established February 2024; headcount approximately 100
Existential Risk Observatory
T2 Nonprofit Governance Active Netherlands
Added as part of the initial AI safety ecosystem sweep. This entry will be tightened and upgraded/dropped based on explicit mission statements and programs in later verification passes.
Profile
ScopeSpecifically prioritises public communication and media outreach as the mechanism to increase societal awareness of x-risk, making it a political priority. Distinct from technical-alignment peers by its communication-science focus and public engagement strategy. Founded in Netherlands in 2021; funded by SFF, LTFF, and AIS Tactical Opportunities Fund.
Programs / outputsPaid AI existential risk research internships (remote, €1,250/month stipend); consensus-building on existential threat models; media tracking and communication campaigns to elevate x-risk in public discourse
FundingSFF; LTFF; private EA donors
Global Partnership on AI (GPAI)
T2 Government Governance Active France
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeInternational governance partnership.
Programs / outputsWorking groups: Responsible AI, Data Governance, Future of Work, Innovation & Commercialisation, AI & Health, AI in Government, AI Compute and the Environment, Generative AI
GPAI Associated Projects (9 planned for 2026): Scaling Responsible AI Solutions, Government Data Sharing Roadmap, VIADUCT, AI@Work Labs Network, Student Communities, Multilingual/Multicultural AI, Living Labs for Impact
Hiroshima AI Process (HAIP) Reporting Framework: transparency reports from organizations developing advanced AI systems
OECD AI Principles and OECD AI Incidents Monitor (AIM)
PublicationsScaling Responsible AI Solutions: Challenges and Opportunities (Dec 2023)
AI for Fair Work Report (Nov 2022)
Data Governance Framework Paper 2.0 (Nov 2022)
Data Justice: A Primer on Data and Economic Justice (Nov 2022)
Partners46 member countries + EU; OECD (hosting Secretariat); CEIMIA (Montreal); Inria (Paris); NICT (Tokyo); G7/Hiroshima AI Process; UNESCO
FundingMember country annual dues of EUR 20,000; integrated partnership with OECD since July 2024; three Centres of Expertise funded by Canada/CEIMIA, France/Inria, Japan/NICT
IEEE SA (Autonomous and Intelligent Systems)
T2 Standards Standards Active United States
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeStandards work for A/IS.
Programs / outputsIEEE 7000 Series: 10+ published ethical standards covering ethical design (7000), transparency (7001), data privacy (7002), algorithmic bias (7003), employer data governance (7005), ontological standard for ethical robotics (7007), fail-safe design (7009), well-being assessment (7010), emulated empathy (7014)
IEEE CertifAIEd: certification program for assessing ethics of autonomous/intelligent systems
Global Initiative 2.0 on Ethics of AIS: focus on Beyond Risk Framing, Safety First Principle, generative AI standards
AI and Ethics in Design: 10-course educational program; Responsible Procurement of AI training framework
PublicationsEthically Aligned Design (EAD) First Edition
IEEE CertifAIEd Ontological Specifications (Ethical Transparency, Algorithmic Bias, Accountability, Privacy)
Trusted Data and AIS Playbook for Financial Services
Children's Data Governance Applied Case Study Report
PartnersOCEANIS (Open Community for Ethics in Autonomous and Intelligent Systems); IEEE 7000 Series developed with hundreds of experts from industry, academia, and government; Industry Connections programs for AI for Public Health, AI in Digital Consumption, Children's Tech Design Governance
FundingIEEE Standards Association industry-funded standards development organization; standards available for purchase; free access program for AI ethics and governance standards
International AI Safety Report (global expert synthesis)
T2 Coalition Mixed Active
The International AI Safety Report is a large multi-author scientific synthesis project reviewing risks and capabilities of general-purpose AI. It is included as an institutional safety knowledge-production initiative rather than a single lab.
Profile
ScopeInternational scientific synthesis of capabilities/risks of general-purpose AI systems.
FundingUK DSIT; G7/G20 member states; AISI network governments
ISO/IEC JTC 1/SC 42 (AI Standards)
T2 Standards Standards Active Switzerland
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeInternational AI standardization committee.
Programs / outputs32 published international standards across 5 Working Groups (Foundational Standards, Data, Trustworthiness, Use Cases, Computational Approaches)
ISO/IEC 42001:2023: world's first AI Management System (AIMS) standard
44 standards under development including generative AI amendments, conformity assessment, human oversight, and red teaming
Joint Working Groups with ISO TC 215 (Health Informatics), IEC SC 65A (Functional Safety), ISO TC 37 (NLP), ISO/IEC JTC 1/SC 7 (Testing)
PublicationsISO/IEC 42001:2023 - AI Management System (AIMS)
ISO/IEC 22989:2022 - AI Concepts and Terminology
ISO/IEC 23894:2023 - Guidance on Risk Management for AI
ISO/IEC 5259 series - Data Quality for Analytics and Machine Learning (Parts 1-5)
Partners70+ liaison organizations including IEEE, ITU, OECD, UNESCO, WEF, WTO, European Commission, ETSI, MLCommons, Partnership on AI, HL7, Cloud Security Alliance; 50+ ISO and IEC technical committees
FundingISO/IEC standards body; funded through member country dues and standard sales; 45 P-members, 25 O-members; ANSI (US) holds secretariat
Japan AI Safety Institute (AISI Japan)
T2 Government Evals Active Japan
AISI Japan is represented here via its published English guidance on AI safety red teaming methodology. This provides strong evidence of safety-evaluation work, though institutional details and mandate should be verified from an official institute overview page.
Profile
ScopePublishes red-teaming methodology guidance on AI safety (documented).
Fieldbuilding / Talent AI safety early-career talent pipeline United States
AI safety fieldbuilding organization announced October 2024; institutional home for SPAR (mentored research program) and FSP. SPAR Spring 2026 ran 130+ projects, the largest AI safety research fellowship round to date.
Profile
Programs / outputsSupervised Program for Alignment Research (SPAR); Fieldbuilder Support Program (FSP); Pathfinder Fellowship
LISA (London Initiative for Safe AI)
T2 Nonprofit Field-building Active United Kingdom
London Initiative for Safe AI, a charity hosting Apollo Research, BlueDot Impact, ARENA, and MATS programs in London.
Profile
Programs / outputsHosting Apollo Research, BlueDot Impact, ARENA, and MATS in London
FundingUK Government; private philanthropists
METR (formerly ARC Evals)
T2 Nonprofit Evals Active United States
METR is the successor name for ARC Evals. Included as a lineage entry; should be merged into the main METR row in canonicalization.
Profile
ScopeModel evaluation and threat research; formerly ARC Evals.
FundingOpen Philanthropy ~$5M+; SFF; ARB; Longview Philanthropy
Mila (Quebec AI Institute)
T2 Academic Technical Active Canada
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeResearch institute with safety-related initiatives.
Programs / outputsAI Safety Studio (launched Oct 2025, guardrails and multi-turn benchmarking, first initiative on mitigating chatbot misuse by users in psychological distress); Canadian AI Safety Institute (CAISI) — 30+ professors and 50+ researchers; International AI Safety Report (chaired by Yoshua Bengio, 96 experts from 30 countries, IPCC model); AI Insights for Policymakers (co-led with CIFAR); Quebec government $36M grant (Feb 2026) for AI research and talent
PartnersCIFAR (co-leading AI Insights for Policymakers); National Research Council Canada; Amii and Vector Institute (Canada's three national AI institutes); UN, OECD, UNESCO (international governance frameworks); UK AISI (International AI Safety Report); 30 countries' expert networks; extensive industry partnerships through Mila's industrial alliance program.
Funding$36M from Quebec government (Feb 2026); CAISI backed by $2.4B federal investment (2024 budget); core funding from CIFAR, NSERC, FRQNT, and provincial sources.
MIRI (Machine Intelligence Research Institute)
T2 Nonprofit Technical Active United States
Long-standing AI safety research organization focused on theoretical alignment, decision theory, and corrigibility research.
Profile
Programs / outputsAI alignment theory, decision theory, corrigibility, logical uncertainty
Mozilla.ai (safety research org)
T2 Nonprofit Technical Active United States (verify)
Mozilla.ai is included as a safety-adjacent research organization referenced by FAR.AI as a collaborator. This row requires direct sourcing from Mozilla.ai’s official materials to confirm scope and programs.
Profile
ScopeTrustworthy, open AI research; safety adjacent.
FundingMozilla Foundation; Mozilla Corporation; specific AI safety research via Mozilla AI Fellowship (~$6M/yr program)
OECD AI Policy Observatory (AI governance)
T2 Government Governance Active France
Added as part of the initial AI safety ecosystem sweep. This entry will be tightened and upgraded/dropped based on explicit mission statements and programs in later verification passes.
Profile
FundingOECD member state contributions; EU Commission co-funding for specific projects
Open Philanthropy (AI risk program)
T2 Nonprofit Field-building Active United States
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeFunder; ecosystem node.
Programs / outputsAI safety grantmaking since 2015, now 'Navigating Transformative AI' (renamed Oct 2025). Technical AI Safety RFP (~$40M over 5 months, 21 research areas in 5 clusters: adversarial ML, sophisticated misbehavior in LLMs, model transparency, trust from first principles, alternative approaches; grants $50K-$5M); AI Governance RFP (closed Jan 2026, $200K-$2M/year); career development and transition funding for AI safety researchers; improving capability evaluations. Estimates 10%+ chance of transformative AI by 2036.
PartnersExtensive grantee network across academic institutions, independent research organizations, and individual researchers. Works closely with Schmidt Sciences, Survival and Flourishing Fund, Foresight Institute. CSET is a major institutional grantee ($97M+ since 2019). Evaluates and funds independently.
Funding~$40M earmarked for Technical AI Safety RFP alone in 2025. Historically: $55M to CSET founding, $8.5M to Epoch AI (2025), $4.13M to Epoch AI (2024), $6.92M to Epoch AI (2023), $8M+ to CAIS. One of the largest AI safety funders globally. Operates as Coefficient Giving for some programs. Total philanthropic funding for AI catastrophic risk mitigation estimated at <$200M/year globally.
Oxford Martin AI Governance Initiative
T2 Academic Governance Active United Kingdom
Oxford Martin AI Governance Initiative is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeIncluded as part of the AI safety ecosystem; mission verification may be needed for safety-first criteria.
FundingJames Martin Foundation endowment; Oxford University budget; UK government research grants
PAI Publication Norms for Responsible AI Workstream
T2 Program Standards Active United States (verify)
A Partnership on AI workstream focused on publication norms for responsible AI research, providing recommendations aimed at mitigating potential harms.
Profile
ScopePublishing norms to mitigate harms and risks from AI research dissemination.
FundingPartnership on AI member dues; Ford Foundation; MacArthur Foundation; tech companies
Coalition Governance Active United States
Added as part of the initial AI safety ecosystem sweep. This entry will be tightened and upgraded/dropped based on explicit mission statements and programs in later verification passes.
Profile
Programs / outputsAI Agents governance (3 major publications on agent governance, real-time failure detection, research agenda); AI Assurance & Accountability Ecosystem (6 publications, closing assurance divide); AI for Human Flourishing (workers' transparency, deepfakes/disclosure, human connection); SAIGE Council + European Steering Committee (launched 2025); Shaping Global Policy (foundation model impact documentation, AI governance toolkit, EU AI Act Code of Practice); Inclusivity and Trust in AI
PublicationsPrioritizing Real-Time Failure Detection in AI Agents; AI Agents & Global Governance; Preparing for AI Agent Governance: Research Agenda; Demand and Incentives for External AI Assurance (Mar 2026); Building Justified Trust in AI Assurers (Mar 2026); Closing the AI Assurance Divide (Feb 2026); Six AI Governance Priorities for 2026 (Feb 2026); 20 new resources published in 2025.
Partners141 partners across 19 countries (2025). Corporate: Adobe, Amazon, Apple, Capital One, Credo AI, DeepMind, EY, Google, IBM, Inflection.ai, Intel, Intuit, JPMorganChase, Mastercard, Meta, Microsoft, OpenAI, Prolific AI, Sony, SAP. 81 convenings in 2025 with 1,530 attendees from 33 countries.
Funding501(c)(3) funded through: (1) General operating funds from Ford Foundation, MacArthur Foundation, Knight Foundation, Luminate Group, Surdna Foundation; (2) Specific project funds; (3) Charitable contributions from 141+ corporate partners (Adobe, Amazon, Apple, Google, Meta, Microsoft, OpenAI, etc.). Specific dollar amounts not publicly disclosed.
Nonprofit Governance Active Netherlands
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeAdvocacy group focused on slowing AI progress until safe.
Programs / outputsInternational advocacy movement for pausing frontier AI development. 15+ protests in 7 countries; PauseCon events (London inaugural, Brussels Dec 2025); National chapter structure (13+ countries, formal MOUs, monthly baseline programming); MicroGrants program; Corporate accountability actions (protests outside Google DeepMind and Anthropic offices); Open letters (60+ UK politicians published in Time); Book-related events for Yudkowsky/Soares 'If Anyone Builds It, Everyone Dies'
PartnersFuture of Life Institute (major funder); Extinction Rebellion-style grassroots organizing model; connections to effective altruism community. Not formally partnered with AI companies (adversarial relationship — protests outside AI company offices).
Funding~EUR 715K total donations (as of Dec 2025). Largest donor: Future of Life Institute (EUR 422,961). Other donors: Greg Colbourn (EUR 95K), Conjointly (EUR 83K), Survival and Flourishing Fund (EUR 9,463), Manifund (EUR 8,221). Current cash on hand: ~EUR 90K. Two paid FTEs; all others volunteers.
RAND Corporation (AI policy / safety research)
T2 Nonprofit Governance Active United States
RAND Corporation (AI policy / safety research) is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeIncluded as part of the AI safety ecosystem; mission verification may be needed for safety-first criteria.
Programs / outputsMeselson Center (biosecurity and AI security within RAND Global and Emerging Risks); Center on AI, Security, and Technology (CAST); Center for the Geopolitics of Artificial General Intelligence; AI Security Guide and Risk Assessment Tool (interactive, Feb 2026, funded by U.S. Department of State); emergency preparedness for AI loss-of-control incidents; governance approaches for securing frontier AI
PublicationsStrengthening Emergency Preparedness and Response for AI Loss of Control Incidents (Jul 2025, 61pp); Governance Approaches to Securing Frontier AI (Oct 2025, 87pp); AI Security Guide and Risk Assessment Tool (Feb 2026); Legal and Policy Approaches to Mitigate Catastrophic Harms from AI (Mar 2026, 43pp, Delphi study with 24 experts); Securing AI Model Weights (May 2024, revised Jun 2024); A Playbook for Securing AI Model Weights (Nov 2024)
PartnersU.S. Department of State (funder of AI Security Guide); U.S. Department of Defense; multiple U.S. government agencies. International partnerships not fully disclosed.
FundingPrimarily U.S. government funded (Department of State, Department of Defense). RAND operates as a nonprofit with $400M+ annual budget across all programs. AI Security Guide funded by U.S. Department of State.
SaferAI Risk Management Ratings
T2 Program Evals Active France
SaferAI’s ratings initiative evaluates frontier AI companies’ risk management practices. Included as a safety governance/evaluations mechanism.
Profile
ScopeCompany risk management practice ratings for frontier AI labs.
FundingSeed funding; commercial revenue from AI risk ratings service
Schmidt Sciences (AI safety support)
T2 Philanthropy Field-building Active United States
Schmidt Sciences is included as an ecosystem funder/collaborator node referenced by FAR.AI. This row should be strengthened by sourcing official funding pages specific to AI safety.
Profile
ScopeFunding/support for safety research (ecosystem node).
Programs / outputsAI2050 Fellowship (4th cohort in 2025, 28 scholars, $18M+ in fellowships, total 99 fellows across 8 countries and 42 institutions since 2022); Science of Trustworthy AI program ($10M launched Feb 2025, 27 projects in first cohort, 3 core aims: characterize/forecast misalignment, develop generalizable measurement/intervention, oversee superhuman-capable AI; 2026 RFP now open)
PartnersAI2050 co-chaired by Eric Schmidt and James Manyika. Science of Trustworthy AI advisory board: Percy Liang, Yonadav Shavit, Ajeya Cotra. Computational support from CAIS. API access from OpenAI. Fellows across 42 institutions globally.
FundingPhilanthropic organization founded by Eric Schmidt. AI2050 has awarded $18M+ in 2025 alone (cumulatively more across 4 cohorts). Science of Trustworthy AI committed $10M in first cohort. Schmidt Futures (now Schmidt Sciences) has committed hundreds of millions across all programs.
Nonprofit Governance Active United States
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeWorks on preventing misuse of advanced AI and strengthening safeguards; mission verification needed.
Programs / outputsNote: 'Secure AI Project' does not correspond to a single well-known distinct organization as of 2025-2026. Multiple entities with 'Secure AI' in their names exist: The Alignment Project by UK AISI (£27M+ for AI alignment research, 60 grantees from 42 countries); ARIA Safeguarded AI programme (£59M for mathematical assurance toolkit for AI); Coalition for Secure AI (CoSAI) under OASIS Open (industry consortium for secure AI standards); MITRE Secure AI project (advancing ATLAS adversarial threat landscape); EU Horizon Europe SecureAI programme (€21M).
FundingGeorgetown University CSET; Open Philanthropy; government grants
Survival and Flourishing Fund
T2 Resource Field-building Active United States
Survival and Flourishing Fund is included as an AI safety ecosystem node. Funding node for long-term survival and flourishing projects (funding). This row is intended for coverage/auditability and may be excluded in a stricter 'orgs only' canonicalization.
Profile
ScopeFunding node for long-term survival and flourishing projects (funding).
Survival and Flourishing Fund (SFF)
T2 Philanthropy / Grantmaking
The Survival and Flourishing Fund (SFF) is an EA-aligned regranting organization focused on reducing existential and catastrophic risks. It operates donor-advised grant rounds that primarily support AI safety organizations, biosecurity, and long-termist causes. SFF acts as an intermediary connecting large EA donors with vetted safety-focused organizations.
Profile
ScopeEA-aligned regranting fund focused on existential risk reduction; primarily funds AI safety, biosecurity, and long-termist cause areas through donor-directed grants.
Programs / outputsS-Process grant recommendation algorithm run 1-2 times per year with independent Recommenders evaluating applications
Initiative Committee making proactive grants without applications (Jaan Tallinn, SFF Advisors, anonymous voters)
Matching Pledges mechanism to leverage outside donations at specified ratios
Cumulative ~$152M in philanthropic gifts organized across history (growing from ~$2M in 2019 to ~$35M in 2025)
PublicationsSFF-2025 S-Process Recommendations Announcement
SFF-2024 Initiative Committee grants announcement
SFF-2024 S-Process Recommendations
Historical recommendations and grant details available at survivalandflourishing.fund/recommendations
PartnersMajor AI safety grantees: CAIS ($1M+ Initiative), MIRI ($1.6M+), Lightcone Infrastructure ($2.3M+), MILA ($4M Initiative), AIPI ($1.9M+), Palisade Research ($1.1M+), CLTR ($565K+), Oxford China Policy Lab ($719K+)
FundingInitially funded by BERI grant from Jaan Tallinn; cumulative ~$152M distributed; SFF-2025: ~$34.92M distributed; SFF-2026 announced: $20-40M estimated
The Alignment Project (UK AISI)
T2 Government Technical Active United Kingdom
The Alignment Project, a £27M+ program under the UK AI Security Institute, funding alignment research across multiple institutions.
Profile
Programs / outputs£27M+ government-funded alignment research program under UK AISI
Funding£27M+ UK Government funding
UN Advisory Body on AI (governance)
T2 Government Governance Active International
UN Advisory Body on AI (governance) is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeIncluded as part of the AI safety ecosystem; mission verification may be needed for safety-first criteria.
FundingUN general budget; voluntary contributions from UN member states
Nonprofit Governance Active United States
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopePublishes analysis/forecasts of AI trajectories; safety-adjacent.
Programs / outputsInteractive model of the future of AI at aifuturesmodel.com (probabilistic forecasting on AI milestones including when AIs achieve human-level coding performance); Substack newsletter; AI scenario analysis. Small research group (5 people: Daniel Kokotajlo, Eli Lifland, Thomas Larsen, Romeo Dean, Lauren Mangla).
FundingNot publicly disclosed. Small independent research group. No major institutional funders publicly listed.
AR
AI Risk and Vulnerability Alliance (ARVA) (bio+AI)
T2 technical-safety Empowering communities to recognize, diagnose, and manage vulnerabilities in general-purpose AI systems active International
AI Risk and Vulnerability Alliance (ARVA) (bio+AI) is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeAdapts cybersecurity CVE (Common Vulnerabilities and Exposures) model to AI systems, prioritising actionable technical fixes over abstract policy. Flagship product AVID is a structured database of AI failure modes, co-developed with academic researchers and cited in regulatory submissions. Distinct from governance peers by empirical, database-driven methodology.
Programs / outputsAI Vulnerability Database (AVID) — open-source knowledge base of AI failure modes; Taxonomy Library for classifying AI risks across model, tool, and application layers
FundingRelated to ARVA; specific BioAI program funding not publicly disclosed
AS
AI Safety Global Society
T2 field-building Reducing existential AI risks through collaborative research, technical upskilling, and advancing impactful careers in AI alignment and interpretability active
Mitigates existential AI risks through community building, professional mentorship, and technical upskilling for early-career computer scientists and engineers transitioning into safety research. Hands-on technical pathway focus distinguishes it from lecture-based fellowship programs.
Profile
ScopeFocuses on hands-on technical upskilling and direct career transition pathways for early-career computer scientists and engineers, with an emphasis on mechanistic interpretability and adversarial ML. Distinct from lecture-based fellowships by providing project mentorship and hackathon-based learning.
Programs / outputsAlignment Research Fellowship (ARF) for technical mentorship; Alignment Jam Hackathons; speaker events, workshops, and social community-building
FundingMembership fees; private donations; EA community fundraising
AS
AI Safety Support (paused)
T2 Nonprofit Field-building Paused
AI Safety Support was a field-building org and resource hub; a public post states it is on indefinite pause. Kept for lineage/history.
Profile
ScopeHistorical field-building and resources (paused).
AT
Alan Turing Institute (AI governance/safety)
T2 Academic Mixed Active United Kingdom
Alan Turing Institute (AI governance/safety) is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeIncluded as part of the AI safety ecosystem; mission verification may be needed for safety-first criteria.
AT
Alan Turing Institute (AI safety interest group)
T2 Academic Mixed Active United Kingdom
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeAI safety interest group page.
AT
Alan Turing Institute (AI Safety)
T2 Academic Research Institute
The Alan Turing Institute is the UK's national institute for data science and artificial intelligence. Its AI safety research covers robustness, interpretability, fairness, and sociotechnical risk. It advises the UK government on AI strategy and runs the AI Safety Hub, coordinating academic safety research across UK universities.
Profile
ScopeUK national institute for data science and AI; conducts safety, robustness, fairness, and governance research. Hosts AI Safety Hub and collaborates with UK government on AI policy.
Programs / outputsCAISI Research Program: $2.4M invested to launch 12 new research projects with 55+ experts across Catalyst Projects and Solution Networks
Data Safe Haven: open-source framework for secure analysis of sensitive data (GitHub: alan-turing-institute/data-safe-haven)
TEA Techniques: interactive database of 100+ responsible AI techniques organized by 7 assurance goals (Trustworthy and Ethical Assurance)
Systemic AI Safety Fast Grants: co-funded with UK AISI and UKRI for systemic approaches to AI safety
PublicationsAI Assurance in Defence: workflow, system card template, and commander's guide (with MoD and Accenture)
CAISI Year in Review (2025)
Vector Institute frontier AI model evaluation study (11 models, 16 benchmarks, open-sourced)
Funding call: Online Learning Courses in Responsible AI (270,000 GBP total)
PartnersGovernment of Canada / ISED; UK AI Safety Institute (AISI); UKRI; Amii, Mila, Vector Institute; National Research Council Canada; IDRC; Ministry of Defence (AI assurance); Accenture; BBC; University of Edinburgh
FundingEPSRC core funding (grants EP/N510129/1, EP/W001381/1, EP/W037211/1, EP/X03870X/1); UKRI Strategic Priorities Fund; Microsoft Azure credits donation; CAISI: $2.4M from Government of Canada/ISED for first year
For-profit Technical Active United Kingdom
This organization appears on multiple curated AI safety maps. It will be upgraded once primary-source mission statements and concrete programs are captured.
Profile
Programs / outputsACE alignment system (patented alignment method); EquitAI (fairness-related alignment); ClassifAI (classification-related alignment); mission-locked charter requiring technology that increases human safety, agency, ability, well-being, self-actualization, and understanding. Video game benchmark demonstration (Sep 2023) showing progress toward safe agentic AI.
PartnersStuart Armstrong is a mentor at Foresight Institute and advisor at AI Safety Camp. Rebecca Gorman is a Fortune 50 AI Innovator and member of Fortune's Founders Forum. Governmental and international body consulting (not specifically named). EnSpire Oxford connection.
FundingPre-seed rounds (amounts not publicly disclosed). Applying for UK government R&D tax credits. Small startup — ~7 employees.
AR
Alignment Research Center
ARC T2 Nonprofit Technical Active United States
Alignment Research Center appears on multiple curated AI safety maps as a technical safety research organization. This entry is included as probable and will be upgraded once a direct official mission page is captured.
Profile
ScopeTechnical alignment/interpretability and related research.
Programs / outputsMatching Sampling Principle (MSP) — central 2025 focus on mechanistic algorithms that outperform random sampling for estimating properties of neural network outputs; Heuristic Explanations (formal/mathematical notions of explanations); Mechanistic Anomaly Detection (MAD); Low Probability Estimation (LPE) for rare catastrophic outputs; Eliciting Latent Knowledge (ELK); Alignment Robustness on OOD inputs. Actively hiring researchers with theoretical backgrounds.
PartnersNot publicly disclosed. Paul Christiano previously worked at OpenAI. ARC Evals (now METR) evaluated models for Anthropic and OpenAI.
FundingNot publicly disclosed. Non-profit research organization. Former evaluations arm (ARC Evals) spun off into METR in late 2023. Has offered research bounties ($5K for matrix completion problems). Likely funded by major AI safety funders in the effective altruism ecosystem.
Nonprofit Mixed Active Israel
This organization appears on multiple curated AI safety maps. It will be upgraded once primary-source mission statements and concrete programs are captured.
Profile
Programs / outputsAI Policy — Standards (NIST AI Security Institute Consortium and ISO, published preprint on differentiating oversight and control, article in AI & Society advocating for independent AI Audit Standards Board); Mathematical AI Safety team (Vanessa Kosoy) spun off into separate organization (COLT 2025 paper, upcoming JMLR paper); Biosecurity (gene synthesis screening, metagenomic monitoring, WHO International Pathogen Surveillance Network member); Salt iodization advocacy in Israel
PartnersNIST AI Security Institute Consortium; ISO (International Standards Organization); RAND (contract work); ARIA (contract work); WHO International Pathogen Surveillance Network (biosecurity); Israeli Knesset and Ministry of Health (salt iodization); ASRA conference (policy sessions).
FundingOpen Philanthropy; SFF; private EA-aligned donors
AI
Amnesty International (AI & Human Rights)
T2 Nonprofit Governance Active United Kingdom
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeHuman rights risks; safety-adjacent.
Programs / outputsAlgorithmic Accountability Toolkit (Dec 2025): comprehensive guide for investigating and challenging state automation, covering scoping, human rights research, algorithmic auditing, advocacy, and strategic litigation
Ban the Scan campaign against facial recognition technology use against protesters and racialized communities
Welfare algorithms investigations: documenting how automated decision-making in social benefits discriminates against marginalized groups (case studies in Denmark, Serbia, France, Sweden, Netherlands, NYC)
Input on ACHPR Study on Human and Peoples' Rights and AI in Africa (May 2025)
PublicationsAlgorithmic Accountability Toolkit (Dec 2025)
The Urgent but Difficult Task of Regulating Artificial Intelligence (Jan 2024)
Gender Equality, the Digital Space and the Age of Artificial Intelligence (2025)
Breaking up with Big Tech: A Human Rights-Based Argument for Tackling Big Tech's Market Power (2025)
PartnersAfrican Commission on Human and Peoples' Rights (ACHPR); UN Working Group on Discrimination against Women and Girls; civil society coalitions across multiple countries
FundingAmnesty International is a global movement funded by millions of individual supporters and some institutional grants; specific AI program funding not itemized
Government Evals Active Australia
Australia's AI Safety Institute, established November 2025 to evaluate AI models and advise on AI safety policy.
Profile
Programs / outputsAI safety evaluations, red-teaming, standards development
FundingAustralian Government funded
BK
Berkman Klein Center (AI governance)
T2 Academic Governance Active United States
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeResearch on technology policy and AI governance.
FundingEthics and Governance of AI Fund $27M (Knight Foundation, Reid Hoffman, Omidyar Network); Harvard institutional funds
BK
Berkman Klein Center (Harvard)
T2 Academic Research Center
The Berkman Klein Center for Internet & Society at Harvard is an interdisciplinary research center studying the internet's impact on society, law, and governance. Its AI-focused work covers AI ethics, accountability, regulation, and the societal effects of automated systems. It co-manages the Ethics and Governance of AI Fund with MIT Media Lab.
Profile
ScopeHarvard internet and society research center; conducts ethics, governance, and law research on AI and digital technologies.
Programs / outputsEthics and Governance of AI Initiative: working with government officials on AI's ethical implications for media, criminal justice, and autonomous vehicles
Artificial Intelligence Initiative: broader examination of AI's impact across domains with large working group
BKC 2025 Action Report: renewed focus on AI research, hiring Chief AI Scientist, launching open-source trust/transparency tools
CLeAR Documentation Framework for AI transparency
PublicationsFramework for AI Transparency: CLeAR Documentation Framework (2024)
Principled Artificial Intelligence: comparing 36 AI principles documents (Jan 2020)
AI & Human Rights: Opportunities & Risks (Sep 2018)
A Harm-Reduction Framework for Algorithmic Fairness (Aug 2018)
PartnersMIT Media Lab (co-anchor of Ethics and Governance of AI Initiative); Knight Foundation; Omidyar Network; Reid Hoffman; William and Flora Hewlett Foundation; Jim Pallotta
FundingEthics and Governance of AI Fund: $27M committed (launched Jan 2017) by Knight Foundation, Omidyar Network, Reid Hoffman, Hewlett Foundation, Jim Pallotta; BKC and MIT Media Lab received $5.9M as academic anchors
CF
Center for Democracy & Technology (AI)
T2 Nonprofit Governance Active United States
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopePolicy and governance of AI risks.
FundingFord Foundation; Open Society Foundations; MacArthur Foundation; tech company grants
CF
Center for Democracy & Technology (CDT) AI
T2 Policy Advocacy / Civil Liberties
The Center for Democracy & Technology (CDT) is a US nonprofit focused on digital rights and free expression. Its AI program addresses surveillance technology, algorithmic decision-making, civil rights in AI-powered systems, and legislative advocacy for AI accountability. CDT regularly engages with US Congress, EU policymakers, and regulatory agencies.
Profile
ScopeUS digital rights nonprofit; AI work focuses on civil rights, surveillance, algorithmic accountability, and legislative engagement on AI governance.
Programs / outputsAI Governance Lab (launched Oct 2023): developing and promoting technically-informed solutions for AI regulation and governance
AI Policy Tracker: searchable database of all CDT AI-related policy positions
Led coalition of 50+ organizations calling on Biden Administration for safe, effective, non-discriminatory federal AI use
Mobilized 85+ civil society groups urging Congress to prioritize civil rights in US AI legislation
PublicationsAssessing AI: Surveying the Spectrum of Approaches to Understanding and Auditing AI Systems (Jan 2025)
AI in Local Government: How Counties & Cities Are Advancing AI Governance (Apr 2025)
To AI or Not To AI: A Practice Guide for Public Agencies (Mar 2025)
Open-Source AI Models Are Not Inherently Security Risks, But Are Integral to Democracy
PartnersNIST (AI Safety Institute Consortium, 200+ organizations); NTIA (open foundation models work); UK AI Safety Summit (civil society delegate); US-EU Trade & Technology Council; Spanish Presidency of EU (AI Act roundtable)
FundingNonpartisan 501(c)(3) nonprofit; AI Governance Fellowship position ($80-115K compensation); specific AI program funding not itemized
CF
Center for Internet and Society (Stanford CIS)
T2 Academic Governance Active United States
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopePolicy work including AI governance.
FundingStanford Law School; tech industry grants; individual donors
CF
Center for Long-Term Resilience (CLTR)
T2 Nonprofit Governance Active United Kingdom
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeCatastrophic risk org with AI relevance.
FundingSFF; Founders Pledge; private donors; no government funding (stated policy)
CF
Centre for International Governance Innovation (CIGI)
T2 Nonprofit Governance Active Canada
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeThink tank work on AI governance.
FundingGovernment of Canada; Government of Ontario; Jim Balsillie endowment
CF
Centre for Long-Term Resilience (CLTR)
T2 Policy Research / X-risk
The Centre for Long-Term Resilience (CLTR) is a UK charity focused on reducing catastrophic and existential risks to humanity, with a particular focus on AI safety and biosecurity. CLTR engages directly with UK government to advance frontier AI safety policies, supports talent pipelines into UK AI safety roles, and produced the 2021 Future Proof report that influenced UK government AI strategy.
Profile
ScopeUK charity focused on reducing catastrophic and existential risks; AI safety work targets UK government policy on frontier AI and biosecurity.
Programs / outputsAI Policy Unit: working with UK Government on frontier AI regulation, open-source misuse risks, and risk management governance
Loss of Control Observatory: prototype to detect and analyse real-world AI control incidents (5x increase in scheming-related incidents found)
Global Risk Index for AI-enabled Biological Tools framework
Policy advocacy for UK AI bill improvements and AI incident reporting regime
Publications5x Increase in Scheming-Related AI Incidents report
How the UK Government Can Govern the Risk of Loss of Control
The Loss of Control Observatory: a prototype to detect real-world AI control incidents
Securing a Seat at the Table: pathways for advancing UK global leadership in frontier AI governance
PartnersUK Government (trusted thought-partner on frontier AI regulation); UK AISI; informed UK Biological Security Strategy implementation; responded to Covid-19 Inquiry
FundingSFF; Founders Pledge; private donors; no government funding (stated policy)
C(
CIGI (Centre for International Governance Innovation)
T2 Policy Research / International Governance
The Centre for International Governance Innovation (CIGI) is a Canadian think tank focused on global governance challenges. Its AI research program addresses international AI governance, cross-border data flows, AI and geopolitics, and emerging technology regulation, with a focus on multilateral cooperation frameworks.
Profile
ScopeCanadian think tank on global governance; AI research covers international AI governance frameworks, data governance, and geopolitical AI competition.
Programs / outputsGlobal AI Risks Initiative: advancing international governance for global AI risks; developing components of an international treaty/framework convention on advanced AI
AI Empowerment for a Prosperous Future: research on Cooperative AI, Causal AI, and Agile AI for policy design and governance
Building Trust in AI: landscape analysis of government AI programs (finding less than 1% have been evaluated)
OpenCanada.org online platform for public policy discussion
PublicationsBuilding Trust in AI: A Landscape Analysis of Government AI Programs (CIGI Paper No. 272, Aaronson)
Scoping AI Governance: A Smarter Tool Kit for Beneficial Applications (CIGI Paper No. 260)
Data Disquiet: Concerns about the Governance of Data for Generative AI (CIGI Paper No. 290)
The Age of AI Nationalism and Its Effects (CIGI Paper No. 306)
PartnersGovernment of Canada; Government of Ontario; Jim Balsillie (founder); openCanada.org community
FundingSupported by Government of Canada, Government of Ontario, and founder Jim Balsillie; independent non-partisan think tank
Coalition Standards Active United States
Coalition for Secure AI (CoSAI) under OASIS Open, developing international AI security standards with 45+ member organizations.
Profile
Programs / outputsCoalition for Secure AI (CoSAI) under OASIS Open, 45+ member organizations developing AI security standards
PartnersGoogle, Microsoft, Amazon, NVIDIA, 45+ orgs
FundingOASIS Open member dues; corporate members include Amazon, Google, Microsoft, Cisco, IBM, Intel
FO
Future of Humanity Institute (historical; discontinued)
T2 Academic Mixed Active United Kingdom
Future of Humanity Institute (historical; discontinued) is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeIncluded as part of the AI safety ecosystem; mission verification may be needed for safety-first criteria.
FundingOpen Philanthropy ~$13.7M cumulative; Open Society Foundations; Future of Life Institute; Skoll Foundation; Humanity Forward Fund. Closed 2024.
JH
Johns Hopkins Center for Health Security (AI misuse work)
T2 Academic Governance Active United States
Johns Hopkins Center for Health Security (AI misuse work) is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeIncluded as part of the AI safety ecosystem; mission verification may be needed for safety-first criteria.
FundingRobert Wood Johnson Foundation; Bloomberg Philanthropies; Open Philanthropy; Johns Hopkins institutional funding
NA
New America (OTI AI)
T2 Nonprofit Governance Active United States
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeAI accountability and governance work.
FundingSchmidt Futures; George Soros/Open Society; MacArthur Foundation; government contracts
NT
Nuclear Threat Initiative (AI risk work)
T2 Nonprofit Governance Active United States
Nuclear Threat Initiative (AI risk work) is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeIncluded as part of the AI safety ecosystem; mission verification may be needed for safety-first criteria.
FundingCarnegie Corporation of New York; MacArthur Foundation; Open Philanthropy; government grants
Standards Governance Active France (OECD)
The OECD AI Principles are an intergovernmental standard promoting trustworthy AI. Included as a governance/standards node within the safety ecosystem.
Profile
ScopeIntergovernmental standard promoting trustworthy AI principles.
FundingOECD member state contributions
SC
Stanford Center for AI Safety (CAIS - Stanford) (verify)
T2 Academic Technical Active United States
Stanford Center for AI Safety (CAIS - Stanford) (verify) is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeIncluded as part of the AI safety ecosystem; mission verification may be needed for safety-first criteria.
SH
Stanford HAI (policy/safety)
T2 Academic Mixed Active United States
Stanford HAI (policy/safety) is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeIncluded as part of the AI safety ecosystem; mission verification may be needed for safety-first criteria.
Programs / outputsHoffman-Yee Grants ($500K year one, $1-2M follow-on, $27.6M distributed to date); Seed Research Grants ($75K each, ~25 grants/year, $12M+ since 2018); Cloud Credit Grants ($1.8M in FY24); Stanford Digital Economy Lab; Center for Research on Foundation Models (CRFM) — Foundation Model Transparency Index, HELM; RAISE Health (with Stanford Medicine); AI4ALL; Congressional Boot Camp on AI (75+ staffers trained); AI Training for Federal Employees (8,000+ registered); 2025 AI Index Report (8th edition)
PublicationsAI Index Report (annual, 8th edition in 2025); Foundation Model Transparency Index (2023, 2024); Considerations for Governing Open Foundation Models (Science); Score Entropy Discrete Diffusion (best paper, ICML 2024); Smart Start (NEJM); Tuning Our Algorithmic Amplifiers (ACM CSCW 2024); Evo 2 genomics foundation model; Biomni biomedical AI agent. 220+ fellows and affiliated faculty.
PartnersAll seven Stanford schools (interdisciplinary); Industrial Affiliates Program (largest at Stanford, new members: American Express, Hanwha Group, LVMH, PwC, SAP); Government (GSA, OMB for federal training); Medical (Stanford Medicine/RAISE Health); Corporate training (Accenture, PepsiCo, EY); Apolitical platform partnership.
FundingFY24 total income: $39.1M (gifts/other: $31.9M, endowment payouts: $4.5M, sponsored research: $2.7M). Total expenditures: $30.7M. Total research grants distributed: $10.377M in FY24. $45M in total funding to faculty since 2019 across all seven Stanford schools.
Nonprofit Governance Active France
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeAI governance think tank.
Programs / outputsThe Athens Roundtable (7 editions): flagship convening bringing together 1,600+ participants from government, industry, civil society, and academia
European AI Governance program: regulatory sandboxes, measurement/benchmarking, GPAI governance, and enforcement in the EU AI Act
Co-led global AI consultation (10,000+ citizens, 200+ experts) informing the Paris AI Action Summit
Co-organized Global Call for AI Red Lines (90+ organizations, 300+ leaders)
PublicationsAhead of the Curve: Governing AI Agents under the EU AI Act
Europe's AI Strategy: Mapping the EU's Emerging AI Policy Portfolio (96 initiatives)
Serious Incident Prevention for AI: lessons from other industries
Heavy is the Head that Wears the Crown: risk-based tiered approach to governing GPAI
PartnersOECD (long-standing collaboration); UNESCO; Harvard Kennedy School; GPAI; Patrick J. McGovern Foundation; Future of Life Institute; IEEE-SA; GIZ; PwC UK; POLITICO; European Parliament; European AI Office; Network of AI Safety Institutes; Agora Strategy Group; Lexxion Publisher
FundingIndependent 501(c)(3) nonprofit; primarily supported by philanthropic organizations; funded in part through donations and service-based contracts with IGOs, governments, and private organizations; publishes Form 990 tax documents
TI
The Institute for AI Policy and Strategy (IAPS)
T2 Nonprofit Training Active United States
The Institute for AI Policy and Strategy (IAPS) is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeIncluded as part of the AI safety ecosystem; mission verification may be needed for safety-first criteria.
FundingOpen Philanthropy; SFF; private EA-aligned donors
UB
UC Berkeley AI Research (BAIR) - safety adjacent
T2 Academic Mixed Active United States
BAIR is an academic AI research umbrella that includes safety-relevant groups such as CHAI. It is included only as an ecosystem linkage node and would typically be excluded under a stricter 'safety-first org' definition.
Profile
ScopeAcademic AI research umbrella; contains safety-aligned groups (e.g., CHAI).
FundingNSF, DARPA, ONR, Berkeley research grants, industry partnerships (Google, Meta, Amazon, Microsoft)
UA
Understanding AI Safety (policy evidence hub)
T2 Coalition Governance Active
Understanding AI Safety is a policy-oriented resource hub emphasizing science- and evidence-based AI policy. It is included as part of the governance ecosystem; details about its organizational structure should be verified.
Profile
ScopeEvidence-based AI policy informed by scientific understanding of AI risks and mitigations.
FundingUK DSIT; UKRI AI Safety Research programme
WE
World Economic Forum (AI)
T2 Nonprofit Governance Active Switzerland
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeAI governance and risk work.
ACTS AI Institute / ACAII (Kenya)
T3 Nonprofit Mixed Active Kenya
African Center for Technology and Society AI Institute, focusing on AI safety evaluation and capacity building in East Africa.
Profile
Programs / outputsAfrican AI safety evaluation, capacity building for AI safety in East Africa
FundingUK DSIT; Open Philanthropy; IDRC; African AI capacity building programs
Agentic Futures Initiative
T3 Nonprofit Governance Active United States
US-based policy initiative focused on governance and regulatory frameworks for agentic AI systems.
Profile
Programs / outputsAgentic AI policy research, regulatory frameworks for autonomous AI systems
FundingEarly stage; SFF; private EA donors
AI Safety Funders Directory (AISafety.com)
T3 Resource Field-building Active
AI Safety Funders Directory (AISafety.com) is included as an AI safety ecosystem node. Directory of funders offering financial support to AI safety projects. This row is intended for coverage/auditability and may be excluded in a stricter 'orgs only' canonicalization.
Profile
ScopeDirectory of funders offering financial support to AI safety projects.
FundingNot applicable — directory website
AI Safety Map (AISafety.com)
T3 Resource Field-building Active
AISafety.com maintains a public map of AI safety organizations. It is included as a meta-resource for coverage tracking, not as a direct safety research/governance organization.
Profile
ScopeIncluded as a meta-resource; not an AI safety org doing safety work itself.
FundingNot applicable — directory website
Alignment Ecosystem Development Discord
T3 Resource Field-building Active
Alignment Ecosystem Development Discord is included as an AI safety ecosystem node. Community infrastructure mentioned as organizer for AISafety.com reading group. This row is intended for coverage/auditability and may be excluded in a stricter 'orgs only' canonicalization.
Profile
ScopeCommunity infrastructure mentioned as organizer for AISafety.com reading group.
FundingCommunity Discord server; no formal funding
For-profit Standards Active United States
CSAI Foundation, announced at RSA 2026, establishing CVE-like authority for agentic AI vulnerabilities.
Profile
Programs / outputsCVE authority for agentic AI, AI vulnerability enumeration and disclosure standards
FundingPhilanthropic donations; early stage
Government Technical Active United States
DARPA SABER program, conducting AI red-teaming and adversarial testing for battlefield and military AI systems.
Profile
Programs / outputsAI red-teaming for battlefield systems, adversarial testing of military AI
FundingDARPA (US DoD) program budget; part of US Defense R&D spending
DNV National Consortium for Safe Industrial AI
T3 Coalition Standards Active Norway
Norwegian consortium led by DNV for developing safety standards and risk assessment frameworks for industrial AI systems.
Profile
Programs / outputsIndustrial AI safety standards, risk assessment frameworks
FundingDNV Group (private Norwegian company); Equinor; Norwegian government innovation grants
For-profit Technical Active United States
US startup applying mechanistic interpretability to AI safety, developing tools for model editing and steering based on interpretability research.
Profile
Programs / outputsInterpretability-based AI safety, model editing and steering, mechanistic interpretability tooling
Government Governance Active Brazil
IBGIA (Instituto Brasileiro de Governança de IA), Brazil's AI governance institute developing regulatory frameworks for AI systems.
Profile
Programs / outputsBrazilian AI governance framework, regulatory policy for AI systems
FundingBrazilian government; FAPESP; CNPq; AI governance capacity building
Nonprofit Technical Active Kenya
Pan-African program developing Africa-centric AI safety evaluation benchmarks and methodologies.
Profile
Programs / outputsPan-African AI safety evaluations, Africa-centric AI safety benchmarks
FundingOpen Philanthropy; SFF; AI governance capacity building
Government Mixed Active India
India's AI Safety Institute, operating as a virtual hub-and-spoke model coordinating AI safety evaluation across Indian research institutions.
Profile
Programs / outputsVirtual hub-and-spoke model for AI safety evaluation across India
FundingGovernment of India (MEITY); nascent organization (announced 2024)
For-profit Technical Active France
AI safety lab with offices in Paris, Mumbai, and London, focused on interpretability and unlearning techniques for AI safety.
Profile
Programs / outputsInterpretability research, machine unlearning techniques
FundingEarly stage; undisclosed
New America OTI (Open Technology Institute)
T3 Policy Research / Digital Rights
New America's Open Technology Institute (OTI) is a technology policy program within the New America think tank. Its AI-relevant work addresses surveillance technology, platform accountability, digital equity, and the civil liberties implications of AI deployment in public services and law enforcement.
Profile
ScopeUS think tank program focused on technology policy; AI work covers surveillance, platform accountability, and equitable access to AI.
Programs / outputsOTI AI Policy work: rights- and risk-based frameworks for AI governance; analysis of Biden EO 14110 and Trump AI Policy Framework
Open-source AI models report: five key attributes of openness for AI models, arguing against broad restrictions on open models
Agentic AI and privacy research (won Privacy Papers for Policymakers Award, Mar 2026)
AI in Public Services research: The Demand Machine (Feb 2026), The AI Lab Next Door (Mar 2026)
PublicationsOpen-Source AI Models Are Not Inherently Security Risks, But Are Integral to Democracy
The Demand Machine: The Realities of AI-Powered Public Service (Feb 2026)
The AI Lab Next Door: Why universities are valuable AI partners for local governments (Mar 2026)
Is There a Third Way for AI, Led by the World's Middle Powers? (Shangri La Series, Mar 2026)
PartnersNTIA (open foundation models recommendations reflected in final report); FCC (cited OTI 68 times in net neutrality reclassification order); 46 organizations on open AI model protections; Tech Policy Press
FundingSchmidt Futures; Soros/Open Society; MacArthur Foundation; government contracts
OpenAI + Apollo scheming evaluations (collaboration node)
T3 Coalition Evals Active International
This row represents a collaboration artifact (OpenAI + Apollo Research on scheming evaluations), not a distinct safety organization. Included only for lineage/attribution tracking.
Profile
ScopeJoint work on scheming evaluations; not a standalone org.
FundingOpenAI $6.6B+ raised (2024); Apollo Research separately funded by Open Philanthropy
Partnership on AI - Safety-Critical AI Program (workstream)
T3 Program Standards Active United States
Partnership on AI - Safety-Critical AI Program (workstream) is included as an AI safety/governance ecosystem organization based on its published AI policy, governance, or safety-related work. It will be upgraded or excluded under a strict safety-first definition after mission verification.
Profile
ScopeIncluded as part of the AI safety ecosystem; mission verification may be needed for safety-first criteria.
FundingPAI member dues; Ford Foundation; MacArthur Foundation; tech companies
For-profit Technical Active United States
CrowdStrike's Project QuiltWorks, using AI to discover and remediate software vulnerabilities at scale.
Profile
Programs / outputsAI-discovered vulnerability remediation, automated security patching
PartnersCrowdStrike
FundingOpen Philanthropy; early stage
Redwood Research (Alignment Forum profile)
T3 Resource Technical Active United States
This is a profile page about Redwood Research, not a distinct organization. Included as a dedupe artifact only.
Profile
ScopeMeta-profile; not distinct from Redwood org (kept for dedupe log).
FundingOpen Philanthropy ~$5M+; SFF; private EA donors
For-profit Technical Active United States
Research organization applying singular learning theory to understand neural network generalization and AI safety.
Profile
Programs / outputsSingular learning theory research, neural network generalization theory
FundingOpen Philanthropy; SFF; LTFF; private EA donors
Volunteer Projects Directory (AISafety.com)
T3 Resource Field-building Active
Volunteer Projects Directory (AISafety.com) is included as an AI safety ecosystem node. Directory to map current AI safety research teams and gaps. This row is intended for coverage/auditability and may be excluded in a stricter 'orgs only' canonicalization.
Profile
ScopeDirectory to map current AI safety research teams and gaps.
FundingNot applicable — directory/listing
WISE (Women in Safety and Ethics)
T3 Nonprofit Field-building Active France
Women in Safety and Ethics, a France-based global community with 1,388 members working to increase diversity in AI safety.
Profile
Programs / outputs1,388-member global community for women in AI safety and ethics
FundingMembership fees; EA community support; undisclosed grants
For-profit Technical Active United Kingdom
UK-based research organization focused on AI control and preventing agentic misalignment in advanced AI systems.
Profile
Programs / outputsAI control research, agentic misalignment prevention
FundingEarly stage; undisclosed
AS
AI Safety Orgs Map (Leo McKeereid)
T3 Resource Field-building Active
A curated AI safety organization map used as a coverage seed resource. Included only as a meta-source node for auditability of the census.
Profile
ScopeMeta-map; not itself doing AI safety work.
FundingNot applicable — individual curation project
Resource Field-building Active United States
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeCommunity forum; meta node.
Programs / outputsCurated archive of ~2,000 posts and ~9,000 comments from AI alignment researchers (launched 2018)
Higher-content-quality sibling of LessWrong with separate reputation system and member-only posting
Serves as primary publication venue for technical AI alignment research proposals and debate
Introductory sequences: AGI Safety from First Principles (Ngo), Value Learning (Shah), Iterated Amplification (Christiano), Embedded Agency (Garrabrant & Demski)
PublicationsAn Overview of 11 Proposals for Building Safe Advanced AI (evhub)
What Is The Alignment Problem? (johnswentworth, Jan 2025)
Risks from Learned Optimization / inner alignment series
AGI Ruin: A List of Lethalities (Yudkowsky)
PartnersLightcone Infrastructure (operator); LessWrong (shared codebase and community); independent oversight board with representatives from major alignment research organizations
FundingOpen Philanthropy; SFF; LTFF; Lightspeed Grants; affiliated with LessWrong/CEA
Academic Standards Active Australia
Tasmania-based research institute focusing on AI safety research infrastructure and evaluation methodology.
Profile
Programs / outputsAI safety research infrastructure, evaluation methodology development
FundingEarly stage; undisclosed
Government Mixed Active China
China's national networked coalition for AI safety development and assessment, established February 2025.
Profile
Programs / outputsNetworked coalition for AI safety development and assessment
FundingChinese government; CAICT; MOST
EA
European AI Alliance
T3 Government Field-building Active Belgium
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeEU community platform; not a dedicated safety org.
FundingEuropean Commission; Horizon 2020/Europe; member state contributions
For-profit Technical Active United Kingdom
UK-based AI safety research organization focused on alignment pretraining and AI control methodology.
Profile
Programs / outputsAlignment pretraining research, AI control techniques
FundingOpen Philanthropy; SFF; early stage
I(
ICAIRE (Saudi Arabia)
T3 Government Governance Active Saudi Arabia
ICAIRE (International Center for AI Research and Ethics), Saudi Arabia's UNESCO-backed center for AI ethics and safety.
Profile
Programs / outputsUNESCO-aligned AI ethics and safety research
FundingSaudi Vision 2030; SDAIA (Saudi Data and AI Authority); government funding
Government Evals Active France
INESIA (Initiative Nationale pour l'Evaluation de la Sécurité de l'Intelligence Artificielle), France's national AI safety evaluation initiative established January 2025.
Profile
Programs / outputsAI safety evaluations, European AI testing infrastructure
FundingFrench government (DINUM); Agence Nationale de la Sécurité des Systèmes d'Information
Government Evals Active Kenya
Africa's first AI Safety Institute, established 2024, focusing on AI safety evaluation with particular attention to African contexts and languages.
Profile
Programs / outputsAI safety evaluations, Africa-first approach to AI safety
FundingKenyan government; UK DSIT collaboration; AISI network participation
Resource Field-building Active United States
Included in Batch 4 to broaden governance/standards/evaluation coverage around AI safety. This entry requires mission verification to determine if it qualifies as safety-first under the strict definition.
Profile
ScopeCommunity platform; meta node.
Programs / outputsCommunity platform for improving reasoning and decision-making with ~3-9x growth in activity metrics since 2018
Annual Review process highlighting best content each year
Primary venue for public thinking on AI existential risk; concepts originating on LW influenced UK government COVID response, UK AI Safety Summit, FTC policy
Lighthaven campus (35,000 sq ft Berkeley venue) for events, fellowships, and conferences; 5,000+ meetups and 50+ events organized
PublicationsThe Sequences / Rationality: From AI to Zombies (Yudkowsky)
Embedded Agency sequence (Garrabrant & Demski)
Harry Potter and the Methods of Rationality (HPMOR)
PartnersCenter for Applied Rationality (CFAR, fiscal sponsor transitioning to independent 501(c)(3)); Lightcone Infrastructure (operator); AI Alignment Forum (sister project)
FundingOpen Philanthropy; LTFF; Center for Applied Rationality; community donations
For-profit Technical Active Kenya
Kenyan AI company developing safety evaluations for African languages and multilingual AI safety benchmarks.
Profile
Programs / outputsAfrican language safety evaluations, multilingual AI safety benchmarks
FundingEarly stage; African AI capacity building; undisclosed
MO
Map of AI Safety v2 (LessWrong post)
T3 Resource Field-building Active
Map of AI Safety v2 (LessWrong post) is included as an AI safety ecosystem node. Meta-post documenting AISafety.com map categories and ecosystem. This row is intended for coverage/auditability and may be excluded in a stricter 'orgs only' canonicalization.
Profile
ScopeMeta-post documenting AISafety.com map categories and ecosystem.
FundingNot applicable — LessWrong post/map
For-profit Field-building Active United Kingdom
London-based organization running 9-week AI safety research fellowships to train and develop alignment researchers.
Profile
Programs / outputs9-week research fellowships in AI safety, alignment researcher training
FundingOpen Philanthropy; SFF; LTFF
SA
Safe AI for Humanity Foundation
T3 Nonprofit Technical Active United States
US nonprofit foundation focused on red-teaming and alignment research to ensure AI systems are safe for humanity.
Profile
Programs / outputsRed-teaming and alignment research, AI safety testing methodologies
FundingPhilanthropic donations; undisclosed
Government Evals Active Singapore
Singapore's AI Safety Institute, established May 2024, focusing on AI model evaluations and regional safety coordination.
Profile
Programs / outputsAI safety evaluations, ASEAN AI safety cooperation
FundingSingapore government (IMDA, MTI); Smart Nation initiative funding
Government Evals Active South Korea
South Korea's AI Safety Institute, established November 2024, focusing on AI model evaluation and safety research.
Profile
Programs / outputsAI safety evaluations, international cooperation on AI safety standards
FundingSouth Korean government (MSIT); Korea AI Safety Institute (announced 2024)
For-profit Technical Active United States
US nonprofit research lab building monitoring and transparency tools for AI systems, focused on dynamic auditing and oversight.
Profile
Programs / outputsAI monitoring, dynamic auditing, model behavior oversight
