Community

Galadriel: The First L1 for AI

Galadriel: The First L1 for AI

Apr 10, 2024

The Creation of a New Species

Today, we are on the cusp of inventing a new species on this planet—one that runs on silicon and, in evolutionary terms, is evolving orders of magnitudes faster than anything else this planet has seen. This can be observed in the 1000x AI compute increase in the past eight years on the left image as well, measured in the model’s capability increase as seen on the right image.

Five years ago, before GPT-3, experts predicted AGI to be 80 years away, and today, the median prediction is 2030. This shows that once AI starts accelerating as it is now, predicting the outcomes is extremely hard because of its exponential nature.

Also, the people at the forefront of AI predict humanity will reach AGI by 2030 or possibly sooner than that, including Sam Altman, Demis Hassabis, Jensen Huang, Dario Amodei, and Elon Musk.

As a team, we’ve been in the AI industry since 2018, and all of this progress is breathtaking and extremely exciting. However, we are increasingly worried because today’s approach to AI safety, ownership, and governance is fundamentally broken. Let’s see why.

“Unfortunately, humanity’s future is in the hands of…”

In 2018, Elon Musk wrote the following letter to the founders of OpenAI.

“Unfortunately, humanity’s future is in the hands of Larry Page (the redacted part).” Elon founded OpenAI as a counterbalance to Google, but unfortunately, the situation is now even worse because of the acceleration risk—a racing dynamic between companies leading to a decline in safety standards, the diffusion of bad norms, and accelerated AI timelines stated by OpenAI. AI, as of today, is controlled, governed, and owned only by a few giant monopolies, including OpenAI, Microsoft, Google, Meta, and a few closed-sourced AI labs. Would you trust your or your family’s lives in the hands of Sam Altman? The man who promises to bring the world a safe AI when he can’t even govern his own company, as shown by the board hiatus. No one is admitting the fact, but today, the fate of 8 billion people is in the hands of less than 50 people who control AI - boards of the aforementioned companies.

As a team, we have worked in cybersecurity, AI safety, and Web3 since 2015,. Throughout our years in cybersecurity, we’ve learned that security through obscurity never works - history has repeatedly shown that. For example, a Chinese spy was recently caught stealing AI secrets from Google. We expect AI risks to have multiple phases. Phase one is about humans using AI for malicious purposes - hacking, disinformation, propagating biases, etc. After that, in phase two, once AI gains autonomy, the risk shifts from humans to AI itself becoming the threat model. Autonomy enables everything from “malware with brains,” which could aim to infect the whole Internet, as there have been examples in the case of Mydoom, which at its peak sent 25% of all emails on the Internet or Zeus, which breached 88% of all Fortune 500 companies. Autonomy will further lead to AI directing its evolution as it could start forking itself through retraining with new data that it could detect to be missing from achieving its goal. Furthermore, there are long-term risks with superintelligence as the control problem is yet to be solved. The reason why we used to work in cybersecurity and NATO is because cybersecurity has never been for us about protecting technical systems but protecting human lives - when critical systems in hospitals or electrical grids are taken down, lives are on the line. Considering the previous, we believe the AI oligopoly’s existing approach to safety is fundamentally flawed, and not learning from history will most likely lead to disastrous results for society.

Besides the lack of safety, another glaring problem is the lack of ownership and governance. In the ’90s, there was a strong push for corporate networks by the likes of Microsoft, AOL, etc., meaning that the Internet would be corporate-owned, taxed, and controlled - imagine the Internet being like the App Store. Today, AI is akin to the 90’s corporate intranets where they live on their permissioned and closed-off islands. AI will be the biggest wealth creator throughout human history, but suppose it is solely owned and governed by a handful of existing Web2 monopolies as of today. In that case, the most likely outcome is extreme wealth disparity because of their extractive economic model. Imagine in 5 years OpenAI’s frontier model being able to conduct 50% of the workforce. OpenAI is predicting outcomes as such in its recent research: GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models. In the next coming years, AI will introduce a large scale workforce dislodge and seen in the chart below.

Exposure to LLMs by industry, from the paper GPTs are GPTs

Furthermore, no right-minded entrepreneur will build today on top of Facebook because it shut down all of the companies on its platform, most famously Zynga - almost all of the large Web2 platforms followed the same model with high take rates and anti-competitive behaviour. In a world where AI is closed and owned by a few oligopolies, there is no opportunity for people to own what they build, receive fair compensation for it, or have a say in its progress.

Safe, citizen owned AI

Until January 3, 2009, there was no secure system in the world from the cybersecurity perspective - not in Google, Meta, or even the NSA itself, which has been hacked multiple times. This changed with the invention of Bitcoin and later Ethereum - the proof is in the pudding as Bitcoin secures $1.3T+ and Ethereum $478B+ as of 13.3.2024. They are the world’s biggest bounties and have withstood penetration tests from individual black hat hackers to nation-states. Humanity has invented an impenetrable system for the first time in computer science history.

Secondly, blockchains can be owned by everyone, which means their ownership scales to tens of millions of users, as in the case of Bitcoin and Ethereum, which leads to a far greater wealth distribution. Contrast this to OpenAI, which latest valuation was $80B, yet none of the value has accrued to the users because it is owned by approximately 600 people, counting the team and investors. We believe, with the right incentives, decentralized AI networks can outcompete today’s closed AI - just like the Internet did for the intranets.

Thirdly, blockchains enable scalable governance. Their governance at times might be messy, as seen by the Bitcoin blocksize wars. Still, it is a far superior governance system where thousands of participants - developers, users, and investors can direct the progress compared to a handful of people. This is crucially important given we are inventing a new species on this planet - everyone who wishes should have an opportunity to have a say in it. Furthermore, contrary to popular belief, the community can effectively govern blockchains to stop illicit activity. Examples range from Bittensor shutting down a subnetwork generating illicit material to Filecoin addressing copyright violations.

Thus, we believe large-scale AI deployment should happen on Web3 rails, which is proven to be the most secure infrastructure to date, enabling user ownership and democratic governance. Our mission is to build a safe, citizen-owned AI.

Ethereum for AI

In 2015, Ethereum showed that the ability to write arbitrary software that can provide trust guarantees turned out to be really powerful, and it became the underpinning of Web3. Today, if you are a developer building AI apps or agents, the only option is to deploy them on a centralized stack. During the last OpenAI Dev Day, many developers felt rug pulled as OpenAI bundled their products into GPTs and an assistant API. All centralized AI providers come with a platform risk for devs because they compete against them as they use their data to build competing apps and train next-frontier models, making dev products irrelevant.

Like Ethereum, which enabled writing smart contracts to build dapps, Galadriel enables developers to build AI apps/agents like smart contracts, which provide trust guarantees and are decentralized. Today, developers understand the value of decentralization, which ensures they have a platform that can’t rug-pull them. This is as valid for building with AI because developers want to be the masters of their destiny. Galadriel’s goal as the first L1 for AI is to become the application layer for AI apps/agents, just like ETH has for d(apps).

Building an L1 for AI is no easy feat because, among other reasons, the AI and Web3 compute paradigms are worlds apart. AI favors centralization as it’s scaled on data centers with high-bandwidth interconnects, whereas Web3 is built on decentralization and verifiability, which introduces a large compute overhead. Furthermore, it seems most are trying to merge these two worlds just because it sounds plausibly good without solving any real problem. There’s a strong need to reason from the first principles to avoid that trap. We initially started building Galadriel as an L2 but realized that vertical integration is necessary to enable fully decentralized & autonomous AI agents/apps. Vitalik initially thought of building smart contracts on Bitcoin but was said no by the core developers. This turned out to be an exceptionally good outcome because Ethereum also had to be vertically integrated to enable smart contracts. Purpose-built tools enable purpose-built use cases, and AI is one of them. Galadriel’s L1 stack is the following:

Now, let’s break it down.

Autonomy

The holy grail of AI will become autonomy. Just like humans evolved a higher level of autonomy than animals, AI’s next step in its evolution is toward more autonomy. The AI labs are already working on that through agentic LLMs (1, 2, 3). This means that users can describe higher-level tasks/goals to AI, and it puts together a plan and executes it step-by-step. In essence, it means we will interact with AI in the next few years, just as with a team member: they will become our software engineers, marketers, accountants, lawyers, etc. Devin, the first AI software engineer, is an excellent example of that. Humans allocate resources based on the complexity and reward of the goals, and the next frontier models will have a dynamic inference budget allocated depending on the complexity of the task. For example, given a mathematical formula to prove, the model could run for hours to find the proof; meanwhile, generating a blog post might take seconds.

Yet, AI can’t achieve full autonomy in the centralized servers of OpenAI, etc, because of the platform risk. We believe that once AI achieves autonomous capabilities, it will trend towards rails, which provide trust guarantees for autonomy. Today, no blockchain natively supports autonomy. We aim to deliver node-level capability for full autonomy, meaning agents can self-execute, fork, interact, trade, etc, without the risk of relying on centralized third parties.

DevExp in Solidity

EVM has the most significant mind share among developers and the biggest network effects. Thus, we aim to provide developers with a frictionless and familiar development experience without having to learn a totally new stack from scratch. This is crucial to letting 1000 flowers blossom and enabling a wide variety of experimentation.

Let’s see an example implementation of an on-chain ChatGPT:

// SPDX-License-Identifier: UNLICENSED
pragma solidity ^0.8.9;

interface IOracle {
    function createLlmCall(
        uint promptId
    ) external returns (uint);
}

contract ChatGpt {
    struct Message {
        string role;
        string content;
    }
  
    struct ChatRun {
        address owner;
        Message[] messages;
        uint messagesCount;
    }
  
    mapping(uint => ChatRun) public chatRuns;
    uint private chatRunsCount;
  
    event ChatCreated(address indexed owner, uint indexed chatId);
  
    address private owner;
    address public oracleAddress;
  
    event OracleAddressUpdated(address indexed newOracleAddress);
  
    constructor(address initialOracleAddress) {
        owner = msg.sender;
        oracleAddress = initialOracleAddress;
        chatRunsCount = 0;
    }
  
    modifier onlyOwner() {
        require(msg.sender == owner, "Caller is not owner");
        _;
    }
  
    modifier onlyOracle() {
        require(msg.sender == oracleAddress, "Caller is not oracle");
        _;
    }
  
    function setOracleAddress(address newOracleAddress) public onlyOwner {
        oracleAddress = newOracleAddress;
        emit OracleAddressUpdated(newOracleAddress);
    }
  
    function startChat(string memory message) public returns (uint i) {
        ChatRun storage run = chatRuns[chatRunsCount];
      
        run.owner = msg.sender;
        Message memory newMessage;
        newMessage.content = message;
        newMessage.role = "user";
        run.messages.push(newMessage);
        run.messagesCount = 1;
      
        uint currentId = chatRunsCount;
        chatRunsCount = chatRunsCount + 1;
      
        IOracle(oracleAddress).createLlmCall(currentId);
        emit ChatCreated(msg.sender, currentId);
      
        return currentId;
    }
  
    function onOracleLlmResponse(
        uint runId,
        string memory response,
        string memory errorMessage
    ) public onlyOracle {
        ChatRun storage run = chatRuns[runId];
        require(
            keccak256(abi.encodePacked(run.messages[run.messagesCount - 1].role)) == keccak256(abi.encodePacked("user")),
            "No message to respond to"
        );
      
        Message memory newMessage;
        newMessage.content = response;
        newMessage.role = "assistant";
        run.messages.push(newMessage);
        run.messagesCount++;
    }
  
    function addMessage(string memory message, uint runId) public {
        ChatRun storage run = chatRuns[runId];
        require(
            keccak256(abi.encodePacked(run.messages[run.messagesCount - 1].role)) == keccak256(abi.encodePacked("assistant")),
            "No response to previous message"
        );
        require(
            run.owner == msg.sender, "Only chat owner can add messages"
        );
      
        Message memory newMessage;
        newMessage.content = message;
        newMessage.role = "user";
        run.messages.push(newMessage);
        run.messagesCount++;
        IOracle(oracleAddress).createLlmCall(runId);
    }
  
    function getMessageHistoryContents(uint chatId) public view returns (string[] memory) {
        string[] memory messages = new string[](chatRuns[chatId].messages.length);
        for (uint i = 0; i < chatRuns[chatId].messages.length; i++) {
            messages[i] = chatRuns[chatId].messages[i].content;
        }
        return messages;
    }
  
    function getMessageHistoryRoles(uint chatId) public view returns (string[] memory) {
        string[] memory roles = new string[](chatRuns[chatId].messages.length);
        for (uint i = 0; i < chatRuns[chatId].messages.length; i++) {
            roles[i] = chatRuns[chatId].messages[i].role;
        }
        return roles;
    }
}

Agent orchestration & tools

AI agents run in loops, which allow them to reason over the goals, break them down into tasks, and decide when to use tools such as a web browser, code interpreter, terminal, etc. All of this is necessary to enable autonomy and highly capable AI agents. We are bringing agent tools on-chain through our teeML implementation. It can run any arbitrary tool in an off-chain oracle, executed in a trusted execution environment (TEE). teeML provides attestations that anybody can verify off-chain. Furthermore, as teeML is open-sourced, developers can freely fork and deploy their own versions.

Encrypted user data

AI is a highly personalized technology that is customized based on data. Thus, we aim to bring user data on-chain in an encrypted manner so the AI apps/agents that developers build can be truly unique and guarantee privacy for end-users. L1 is as valuable as much as the heterogenous state it provides; this is especially true in the case where data is one of the core primitives of enabling the use cases, as is the case with AI. Besides user data, AI agents/apps also need to have their state as embeddings for long-term memory. Enabling encrypted user data is still an active research area and part of our longer-term roadmap. Initially, we target use cases where all the data is public, akin to existing L1s.

Verifiable AI inference

Problem

How do you bring model inference on-chain? There are three major problems to overcome.

  1. LLMs and similar models require a huge amount of FLOPS. Even though on the edge inference is becoming feasible thanks to quantization and hardware improvements, the largest models still need multi-GPU setups for inference.

  2. The latency of LLMs significantly exceeds block time (e.g. 12 seconds on Ethereum). One call to GPT-4 can easily take 20 seconds, and an agent running tens of iterations can take total latency to several minutes.

  3. LLMs are not always deterministic, and even if deterministic LLM inference is possible (with some loss in speed), it is impossible to get consensus on calling external APIs.

The solution space for making AI inference verifiable on-chain is wide and all of them come with trade-offs. Let’s break down the existing solutions.

zkML

Zero-knowledge machine learning (zkML) enables the ML models to be converted into ZK circuits, and for every inference, a proof is generated, which can be later verified on-chain. It provides the highest security guarantees as it relies on cryptographic security, but the downside is a 180x compute overhead. For example, if it takes 0,5 seconds to run a random forest, then it takes 90 seconds to generate the proof. In the case of larger models such as LLMs, it’s simply unfeasible because the proving time grows exponentially with the size of the model parameters. This means that both latency and cost-wise, zkML is not yet ready for its prime time. In the long term, we believe that zkML will have order-of-magnitude improvements and become efficient as zkML libraries are improved and specialized hardware is developed. Vitalik Buterin has also written about this.

opML

Optimistic machine learning (opML) enables off-chain inference using the optimistic approach with an on-chain dispute engine, akin to optimistic rollups on L2s. opML complies ML models to a specialized VM instruction set, which is then run in its VM. The input and output are posted on chain, which another party can then challenge. The challenging party provides a pinpoint where, in the computation graph of the neural net, the result is false. This pinpoint is then sent to an on-chain smart contract, which provides arbitration. Thus, opML is much faster than zkML, but there is always a delay for the challenge period; the models must be deterministic, and the input data is always public.

opML + zkML

What if you could combine opML and zkML? Ora recently introduced optimistic privacy-preserving AI (opp/ai), which aims to combine the privacy-preserving capabilities of zkML with the computational efficiency of opML. In a nutshell, only the first few layers of the neural net are run in zkML, and the rest of the model is run in opML. This provides model privacy while reducing ZK overhead and, theoretically, data privacy if the ZK proof is generated on the client side. A simple MNIST CNN achieves an 80% reduction in proof generation compared to running the whole model through a zkML circuit. But even in this case, the proof generation takes 36 seconds on a beefy VM with 512GB of RAM. In the case of Stable Diffusion, the most popular open-source image generation model, it would still take 19-180TB of RAM and 2-17 days to generate proofs for only the attention layers of the model. Combining opML and zkML is a promising avenue, but an order-of-magnitude improvement in cost and latency is still needed. Also, it can only work with deterministic models, which inhibits its adoption because GPT-4, Mixtral, etc., are all MoE (mixture of expert) models, which makes them non-deterministic.

FHE

Fully homomorphic execution enables data to stay private throughout the computation, making it desirable for AI use cases. However, the current overhead of FHE compute is many orders of magnitude above what is acceptable for AI use cases.

Trusted execution environment

All of the above solutions are developing fast but still infeasible today for frontier LLMs: the overhead of native execution, zkML and FHE is too large and opML can only run deterministic models. However, there is an alternative that enables building on-chain AI already today: running an async oracle in a TEE. This solves all three problems mentioned above:

  1. The large compute requirement is solved by moving the compute off-chain and into a trusted third party service;

  2. Latency is solved by running the oracle async (the request and response are in different blocks), similar to ChainLink oracles;

  3. Consensus is not required, because the verifiability is provided by the trusted enclave, which produces attestations of every execution on-chain. Only trust in the enclave hardware manufacturer is required.

As the first step, we aim to bring verifiable inference on-chain, which is low latency and cost-competitive. The reasoning is that initially, we don’t expect that most AI apps or agents require high computation integrity, which would, for example, be necessary in the case of enabling DeFi with ML. Instead, low latency, throughput, and cost are much more critical. To achieve those constraints, we developed teeML: Trusted Execution Environment for Machine Learning Inference. It enables the querying of any of the existing LLM APIs, both for open and closed-source models, in a low latency, low cost, and verifiable manner. teeML node is run off-chain along with an on-chain oracle contract. Developers only need to call out the oracle contract, and a result is returned to them with a call-back. The teeML generates an attestation of the correct code execution, which anybody can verify off-chain.

teeML execution flow

Trusted execution environments (TEEs) have also been recently proposed to solve Ethereum’s MEV by producing ETH blocks in TEEs, which show its promise in blockchain applications. teeML’s strengths are low latency, cost, and high throughput. On the downside, enclaves can be potentially exploited by a well funded adversary and attacks along with mitigation techniques have been published. As well, enclave execution can’t be fully verified on-chain, thus it’s not a fully trustless system. We at Galadriel run the teeML oracle, which means it’s akin to a centralized sequencer in the case of L2s. For a more trustless setup, developers can also fork or run their own teeML oracle if they choose to. We also aim to incentivize more oracles to provide on-chain inference.


Validator level inference

teeML is the first step towards progressive decentralization and as part of our roadmap, we are planning to develop a fully verifiable AI inference on the node level that can run any open-source model. This meets the needs of securing high TVL use cases as it is trustless and is part of the network consensus. We will be writing more about this in the future.

Low-latency, low-cost, parallel execution EVM

Imagine using AI applications as an end-user and waiting for 10+ seconds. Most users will churn on this because it’s simply bad UX. As many AI apps/agents are end-user-focused, they have to be able to respond with low latency. Furthermore, when agents call out each other, i.e., imagine an agent swarm with 100 agents combined to achieve a goal, it can’t have an order of magnitude overhead from cost or time-wise because otherwise, it can’t compete with centralized solutions. Blockchains introduce cost and latency overhead, and we aim to reduce that overhead for AI agents/apps as much as possible. Thus, we are building Galadriel on a parallel execution EVM stack, specifically off Cosmos-SDK and Sei v2.

Safety & Governance

We believe prioritizing AI safety and governance is not only a morally right thing to do stemming our mission but we also believe that in the upcoming years market will put a premium on it. Regulation will likely try to enforce it and society will not tolerate technology that has a risk to human lives - just see how market reacted to Boeing’s two fatal incidents in 2020.

Challenges

Besides centralized AI, building on-chain AI has it’s own set of challenges:

  • In a fork, any safety system can be modified or removed, making the fork more dangerous;

  • Since all safety mechanisms are public, they can be adversarially optimized against without any public trace (e.g. developing adversarial examples);

  • Standard anti-abuse mechanisms like staking and slashing are harder to use because the abusive behaviour is harder to define objectively (what constitutes a malicious LLM reply?);

  • It is hard to coordinate between the mostly pseudonymous actors in the community;

  • How to shut down agents that fork themselves and become malicious in a decentralized setting.

Core theses

Before we delve into the solution space, we’d like to bring out some of our core theses which impact the design space of safety & governance:

  • Mustafa Suleyman, co-founder of Deepmind has stated that technology always diffuses throughout the world and the same will happen with AGI. There’s just a time delay between the parties who gain access first and who later. This can also be observed with LLMs where it took open-source just over a year to catch up to GPT-4 with the release of Mixtral. Thus we operate on the premise that AGI will eventually diffuse and become widely available;

  • In the 1980s, PCs became useful as standalone devices, but their true value was unlocked by networking them together. Likewise, AI models today are standalone, but their true value will unlock through a network of AIs. AI will gravitate towards the network with the highest utility and access to FLOPS and data. See more in the agent swarms section;

  • AI risks have three distinct phases. Phase one is about humans using AI for malicious purposes - hacking, disinformation, propagating biases, etc. After that, in phase two, once AI gains autonomy, the risk shifts from humans to AI itself becoming the threat model. Thirdly, once superintelligence is achieved it’s a black hole and the outcome is unknown;

  • AI will become destructive because history has shown that it is in human nature that if a technology can be weaponized, then it will be. Meaning, AI itself might be harmless but humans themselves will use it for malicious purposes.

Solutions

The L1 design should incorporate game theory for safety and minimize harmful use-cases. There is no single silver bullet for aligning AI today but we see many promising avenues which we aim to pursue:

  • L1 governing laws could be based upon United Nations Declaration of Human Rights in order not to harm humans. This is inspired by Antrophic’s research in constitutional AI;

  • The L1 game theory should be designed to protect against external adversaries i.e. attacks against the chain itself and also internal adversaries i.e. destructive AI agents which attack the external world;

  • Network could have a mechanism to vote to shut off malicious agents. The voting could also be done potentially by other agents which provide a proof of malicious behaviour and collect a reward for it;

  • Compared to black boxes such as OpenAI, etc. servers, Web3 rails provide transparency. This is crucial in building safe and credible oversight once developers start building and deploying millions of AI agents. In case anything starts going haywire it can be acted upon quickly but the same can’t be said about the servers of OpenAI;

  • Governance could have a kill switch. Network participants could have a method to shut down the network if the network becomes destructive for the society and if they choose to vote upon it. A natural kill switch has also been proposed by Vitalik Buterin;

  • The most cyber-secure systems (BTC, ETH, Linux, etc.) in the world have 4 core properties: they are open-source, they are game theoretic & driven by incentives, they operate in a constant adversarial setting and they leverage wisdom of the crowd. Galadriel’s design is based upon those four core properties.

In the last ten years, applied cryptography has blossomed - everything from FHE to ZK, which was thought to be much farther away. This is mainly due to Ethereum and other L1 ecosystems, which have incentives to invent better crypto primitives and deploy them into production. Alike, Galadriel could become the world’s biggest applied AI safety network, where crypto incentives drive researchers and engineers to align AI. We believe deploying AI at scale on Web3 rails which is constantly battle-tested and improved by thousands of open-source participants will lead to the most secure rails for AI. We will hire and be the first Web3 network with an AI safety team. Its goal is to develop a robust methodology for measuring and progressing AI’s alignment on Galadriel.

Comparison with existing decentralized AI projects

Today, AIxWeb3 is still extremely small, and we believe a rising tide raises all boats. AI works off three core primitives: FLOPS, data, and human coordination—all of those core components have to be built up with crypto-native incentives. We are only scratching the surface, and the infrastructure buildout has only started, which is necessary to train, infer, and deploy AI at scale on Web3 rails. Here’s how Galadriel compares to the existing Web3xAI projects.

We aim to collaborate with many and believe all of the projects are necessary for the Web3xAI infrastructure’s buildout. Compared to others, Galadriel is the first L1 for AI that enables AI-native applications/agents the same way Ethereum enabled dApps.

Use cases

We expect many of the initial use cases to be skeuomorphic as developers start experimenting in this new design space where they can build Web3xAI native apps for the first time—our goal is to enable 1000 flowers to blossom. As it was hard for Vitalik Buterin to predict all of the use cases in 2014 with Ethereum, we are conscious that most use cases today have yet to be invented. Yet, here are some of the potential ones we foresee.

Agent swarms

It’s well-researched that multi-agent systems in complex tasks that require a set of fundamental abilities such as reasoning, multi-modality handling, web browsing, and general tool use outperform frontier LLMs on their own. On GAIA: A Benchmark for General AI Assistants by Meta, Hugging Face, etc., a multi-agent system performs almost 5x better than GPT4-Turbo, as seen on the GAIA Leaderboard.

Thus, Agent swarms can accomplish far more complex tasks when networked together. Swarm intelligence can also be observed in nature, such as in the case of bees, ants, and even humans. Large groups of relatively simple organisms can compute the solutions to complex problems by collectively organizing their behavior in such a way that the actions of one individual do not make much sense, but when put in the context of thousands of other individuals all following the same rules, complex computation takes place. Swarm intelligence follows a given set of steps in social animals, which shows the mechanisms at play in this type of meta-cognition:

  1. Information sharing: the information that each member obtains in its search needs to be shared with the whole swarm so that a decision can be made.

  2. Evaluation: a method for evaluating the information that has been shared must exit so that the swarm can make meaningful decisions. In the case of the honeybee, this method is built in how bees choose which dancing bees to follow. If there are several dancing bees, which one should each of the other bees follow into the new home? The answer is simple: the bee whose dance is more enthusiastic or convincing. Different bees seem to have various ‘opinions’ as to which of their mates makes a better ‘speech,’ giving rise to a sort of ‘elections’ in which there is no absolute consensus.

  3. Consensus building: In the case of bees, when a consensus has not yet been reached, and the difference in the ‘voting outcome’ is not that big (55% versus 45%), a ‘debate’ takes place. The bees start making loud noises and bumping against bees who have a different view from their own. In this manner, some of the bees that receive this type of treatment, as it is uncomfortable for them, change their mind about who to follow in favor of the view of the bee that is ‘confronting’ it. By this method, the swarm avoids deadlock between competing alternatives and ensures that the option that is to be chosen by the majority of the bees is indeed the right one.

  4. Committing on the vote: Now that the colony has made a clear decision by majority vote, the whole swarm follows the winning bee to their new hive.

Today, multi-agent systems follow similar mechanics, and Web3 provides them native rails to enable swarm intelligence - like the DeFi Lego blocks moment where simple core primitives have enabled developers to build a complex interlocking financial system. In the long-term, there’s also a fundamental question of whether AGI will be reached through a singleton, i.e., a single model, or through sorts of swarm intelligence that can be observed through evolution. The existing research seems to point to the latter. We believe Galadriel can significantly contribute to finding that out as we plan to scale on-chain swarm intelligence to millions of interlocking and interacting agents. This, in turn, changes the L1 from a deterministic state machine to more of a living organism driven by adaption - the hallmark of intelligence.

Culture/Social

We loved when NFTs hit the mainstream because they brought culture on-chain and showed that Web3 can be more than just serious financial applications. AI opens up a new experimentation canvas for artists and developers alike. For example, imagine generative NFTs that are constantly changing and evolving. Or, imagine your favorite meme coin becoming alive and having a much deeper connection with it - might it be dogwifhat, Pepe, or Doge.

On-chain Midjourney: https://minter.galadriel.com/
We also imagine use cases where existing centralized products will be decentralized, which can lead to better products with the right incentives. For example, in the same way, Farcaster is building a user-owned social media platform, we envision developers building user-owned ChatGPT, Midjourney, Character.ai, etc.

On-chain ChatGPT:https://chatgpt.galadriel.com/

AI Gaming & Simulations

Imagine a world where AI agents live and interact with each other in a virtual environment, learning and evolving just as humans do. Imagine playing this game where the world is ever-evolving. Google created this kind of Sims-like game in a small scale and we envision developers to scale these sorts of simulations up to millions of agents inhibiting complex behaviours.

We also see games where AI agents compete against each other, and users can bet on winners, like hamsters.gg, or users can compete themselves with on-chain AI to win prizes. As an example, we built VitAIlik: an RPG game where users can battle with an on-chain AI version of Vitalik. Every game is new, as AI generates the lore and gameplay dynamically. Much more of future gaming will be dynamically generated by AI based on the users’ preferences, and we see many of them coming on-chain.

On-chain VitAIlik game:https://vitailik.galadriel.com/

Finance & Settlement Layer for AI

We believe, in the long term, AI will drive GDP growth in the world most through its ability to conduct business between companies on a global scale. Effectively, companies are going from operating at a human speed to machine speed. It’s like the Internet made information move at the speed of light, but now it’s for whole companies. For this, a native payment and settlement layer is necessary where AI agents can exchange money in a trustless manner for the exchanged services.

Secondly, imagine you are part of a DAO but the managing of the treasury is not trustless. We’ve seen the exact problem in many DAOs and this could for example be solved by an AI DAO.

An AI DAO where the treasury is managed by AI agents

Furthermore, we see endless opportunities where AI can improve today’s Web3 products or create new categories by giving AI money and thus increasing its utility:

  • Automated yield farming;

  • Agent-based prediction markets;

  • On-chain insurance run by AI;

  • Hedge funds managed by AI;

Let 1000 flowers blossom

As Vitalik Buterin couldn’t foresee many of today’s use cases, such as NFTs, we imagine most of the use cases are yet to be dreamed of and invented. That’s why our goal is to enable developers to play with this new design space and experiment easily so that 1000 flowers can blossom.

Roadmap

Move fast, ship to users

We as a team have gone with multiple AI & Web3 startups from 0 → 1 and afterwards helped to scale to unicorns. Thus, we believe in the ethos of moving fast and iterating against users vs trying to come up with a perfect solution without grounding in reality. Thus, we err on the side of shipping early which also means if you are developing on top of Galadriel during devnet expect things to evolve and change rapidly.

Research

Besides, the core product there’s fundamental research to be conducted on the following areas: enabling privacy for user data, scalability in verified AI inference, bringing embeddings on chain, AI safety research and more.

Devnet

Galadriel’s Devnet is out now. During devnet developers can build working on-chain AI agents and applications. The focus is on developing the core technology, quickly iterating the product along with user needs and progressively decentralizing. We expect to make many big architectural changes during the devnet as the full chain spec is yet to be implemented.

Testnet

After the validation of the design and architecture during devnet, we aim to lock the chain spec and release a testnet. Biggest upgrade will be in bringing AI inference on the validator level so the network can achieve highest security and verifiability guarantees. It will act as the final testing ground before mainnet together with a security audit. We seek to reward and incentivise the early adopters and community members.

Mainnet

Following the successful iterations and invaluable feedback garnered through our testnet phase, we will launch Galadriel’s mainnet. In line with our tradition of rewarding early adopters and community members, the mainnet launch will distribute incentives and introduce new reward mechanisms.

Vision of Ethereum for AI

Galadriel’s mission is to build safe, citizen owned AI by building ETH for AI. Our long-term vision can be unpacked as:

  1. Build the first L1 for AI which acts as the settlement layer for AI, driving real GDP growth.

  2. Provide direct ownership and democratic governance over AI.

  3. Become the most secure rails for AI deployment and largest applied safety network where crypto-incentives drive humanity to align AI.

What is the future of AI? Will it be closed, corporate-owned, or safe and user-owned? Become a Galadrian and join the mission in making the latter happen.

The Creation of a New Species

Today, we are on the cusp of inventing a new species on this planet—one that runs on silicon and, in evolutionary terms, is evolving orders of magnitudes faster than anything else this planet has seen. This can be observed in the 1000x AI compute increase in the past eight years on the left image as well, measured in the model’s capability increase as seen on the right image.

Five years ago, before GPT-3, experts predicted AGI to be 80 years away, and today, the median prediction is 2030. This shows that once AI starts accelerating as it is now, predicting the outcomes is extremely hard because of its exponential nature.

Also, the people at the forefront of AI predict humanity will reach AGI by 2030 or possibly sooner than that, including Sam Altman, Demis Hassabis, Jensen Huang, Dario Amodei, and Elon Musk.

As a team, we’ve been in the AI industry since 2018, and all of this progress is breathtaking and extremely exciting. However, we are increasingly worried because today’s approach to AI safety, ownership, and governance is fundamentally broken. Let’s see why.

“Unfortunately, humanity’s future is in the hands of…”

In 2018, Elon Musk wrote the following letter to the founders of OpenAI.

“Unfortunately, humanity’s future is in the hands of Larry Page (the redacted part).” Elon founded OpenAI as a counterbalance to Google, but unfortunately, the situation is now even worse because of the acceleration risk—a racing dynamic between companies leading to a decline in safety standards, the diffusion of bad norms, and accelerated AI timelines stated by OpenAI. AI, as of today, is controlled, governed, and owned only by a few giant monopolies, including OpenAI, Microsoft, Google, Meta, and a few closed-sourced AI labs. Would you trust your or your family’s lives in the hands of Sam Altman? The man who promises to bring the world a safe AI when he can’t even govern his own company, as shown by the board hiatus. No one is admitting the fact, but today, the fate of 8 billion people is in the hands of less than 50 people who control AI - boards of the aforementioned companies.

As a team, we have worked in cybersecurity, AI safety, and Web3 since 2015,. Throughout our years in cybersecurity, we’ve learned that security through obscurity never works - history has repeatedly shown that. For example, a Chinese spy was recently caught stealing AI secrets from Google. We expect AI risks to have multiple phases. Phase one is about humans using AI for malicious purposes - hacking, disinformation, propagating biases, etc. After that, in phase two, once AI gains autonomy, the risk shifts from humans to AI itself becoming the threat model. Autonomy enables everything from “malware with brains,” which could aim to infect the whole Internet, as there have been examples in the case of Mydoom, which at its peak sent 25% of all emails on the Internet or Zeus, which breached 88% of all Fortune 500 companies. Autonomy will further lead to AI directing its evolution as it could start forking itself through retraining with new data that it could detect to be missing from achieving its goal. Furthermore, there are long-term risks with superintelligence as the control problem is yet to be solved. The reason why we used to work in cybersecurity and NATO is because cybersecurity has never been for us about protecting technical systems but protecting human lives - when critical systems in hospitals or electrical grids are taken down, lives are on the line. Considering the previous, we believe the AI oligopoly’s existing approach to safety is fundamentally flawed, and not learning from history will most likely lead to disastrous results for society.

Besides the lack of safety, another glaring problem is the lack of ownership and governance. In the ’90s, there was a strong push for corporate networks by the likes of Microsoft, AOL, etc., meaning that the Internet would be corporate-owned, taxed, and controlled - imagine the Internet being like the App Store. Today, AI is akin to the 90’s corporate intranets where they live on their permissioned and closed-off islands. AI will be the biggest wealth creator throughout human history, but suppose it is solely owned and governed by a handful of existing Web2 monopolies as of today. In that case, the most likely outcome is extreme wealth disparity because of their extractive economic model. Imagine in 5 years OpenAI’s frontier model being able to conduct 50% of the workforce. OpenAI is predicting outcomes as such in its recent research: GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models. In the next coming years, AI will introduce a large scale workforce dislodge and seen in the chart below.

Exposure to LLMs by industry, from the paper GPTs are GPTs

Furthermore, no right-minded entrepreneur will build today on top of Facebook because it shut down all of the companies on its platform, most famously Zynga - almost all of the large Web2 platforms followed the same model with high take rates and anti-competitive behaviour. In a world where AI is closed and owned by a few oligopolies, there is no opportunity for people to own what they build, receive fair compensation for it, or have a say in its progress.

Safe, citizen owned AI

Until January 3, 2009, there was no secure system in the world from the cybersecurity perspective - not in Google, Meta, or even the NSA itself, which has been hacked multiple times. This changed with the invention of Bitcoin and later Ethereum - the proof is in the pudding as Bitcoin secures $1.3T+ and Ethereum $478B+ as of 13.3.2024. They are the world’s biggest bounties and have withstood penetration tests from individual black hat hackers to nation-states. Humanity has invented an impenetrable system for the first time in computer science history.

Secondly, blockchains can be owned by everyone, which means their ownership scales to tens of millions of users, as in the case of Bitcoin and Ethereum, which leads to a far greater wealth distribution. Contrast this to OpenAI, which latest valuation was $80B, yet none of the value has accrued to the users because it is owned by approximately 600 people, counting the team and investors. We believe, with the right incentives, decentralized AI networks can outcompete today’s closed AI - just like the Internet did for the intranets.

Thirdly, blockchains enable scalable governance. Their governance at times might be messy, as seen by the Bitcoin blocksize wars. Still, it is a far superior governance system where thousands of participants - developers, users, and investors can direct the progress compared to a handful of people. This is crucially important given we are inventing a new species on this planet - everyone who wishes should have an opportunity to have a say in it. Furthermore, contrary to popular belief, the community can effectively govern blockchains to stop illicit activity. Examples range from Bittensor shutting down a subnetwork generating illicit material to Filecoin addressing copyright violations.

Thus, we believe large-scale AI deployment should happen on Web3 rails, which is proven to be the most secure infrastructure to date, enabling user ownership and democratic governance. Our mission is to build a safe, citizen-owned AI.

Ethereum for AI

In 2015, Ethereum showed that the ability to write arbitrary software that can provide trust guarantees turned out to be really powerful, and it became the underpinning of Web3. Today, if you are a developer building AI apps or agents, the only option is to deploy them on a centralized stack. During the last OpenAI Dev Day, many developers felt rug pulled as OpenAI bundled their products into GPTs and an assistant API. All centralized AI providers come with a platform risk for devs because they compete against them as they use their data to build competing apps and train next-frontier models, making dev products irrelevant.

Like Ethereum, which enabled writing smart contracts to build dapps, Galadriel enables developers to build AI apps/agents like smart contracts, which provide trust guarantees and are decentralized. Today, developers understand the value of decentralization, which ensures they have a platform that can’t rug-pull them. This is as valid for building with AI because developers want to be the masters of their destiny. Galadriel’s goal as the first L1 for AI is to become the application layer for AI apps/agents, just like ETH has for d(apps).

Building an L1 for AI is no easy feat because, among other reasons, the AI and Web3 compute paradigms are worlds apart. AI favors centralization as it’s scaled on data centers with high-bandwidth interconnects, whereas Web3 is built on decentralization and verifiability, which introduces a large compute overhead. Furthermore, it seems most are trying to merge these two worlds just because it sounds plausibly good without solving any real problem. There’s a strong need to reason from the first principles to avoid that trap. We initially started building Galadriel as an L2 but realized that vertical integration is necessary to enable fully decentralized & autonomous AI agents/apps. Vitalik initially thought of building smart contracts on Bitcoin but was said no by the core developers. This turned out to be an exceptionally good outcome because Ethereum also had to be vertically integrated to enable smart contracts. Purpose-built tools enable purpose-built use cases, and AI is one of them. Galadriel’s L1 stack is the following:

Now, let’s break it down.

Autonomy

The holy grail of AI will become autonomy. Just like humans evolved a higher level of autonomy than animals, AI’s next step in its evolution is toward more autonomy. The AI labs are already working on that through agentic LLMs (1, 2, 3). This means that users can describe higher-level tasks/goals to AI, and it puts together a plan and executes it step-by-step. In essence, it means we will interact with AI in the next few years, just as with a team member: they will become our software engineers, marketers, accountants, lawyers, etc. Devin, the first AI software engineer, is an excellent example of that. Humans allocate resources based on the complexity and reward of the goals, and the next frontier models will have a dynamic inference budget allocated depending on the complexity of the task. For example, given a mathematical formula to prove, the model could run for hours to find the proof; meanwhile, generating a blog post might take seconds.

Yet, AI can’t achieve full autonomy in the centralized servers of OpenAI, etc, because of the platform risk. We believe that once AI achieves autonomous capabilities, it will trend towards rails, which provide trust guarantees for autonomy. Today, no blockchain natively supports autonomy. We aim to deliver node-level capability for full autonomy, meaning agents can self-execute, fork, interact, trade, etc, without the risk of relying on centralized third parties.

DevExp in Solidity

EVM has the most significant mind share among developers and the biggest network effects. Thus, we aim to provide developers with a frictionless and familiar development experience without having to learn a totally new stack from scratch. This is crucial to letting 1000 flowers blossom and enabling a wide variety of experimentation.

Let’s see an example implementation of an on-chain ChatGPT:

// SPDX-License-Identifier: UNLICENSED
pragma solidity ^0.8.9;

interface IOracle {
    function createLlmCall(
        uint promptId
    ) external returns (uint);
}

contract ChatGpt {
    struct Message {
        string role;
        string content;
    }
  
    struct ChatRun {
        address owner;
        Message[] messages;
        uint messagesCount;
    }
  
    mapping(uint => ChatRun) public chatRuns;
    uint private chatRunsCount;
  
    event ChatCreated(address indexed owner, uint indexed chatId);
  
    address private owner;
    address public oracleAddress;
  
    event OracleAddressUpdated(address indexed newOracleAddress);
  
    constructor(address initialOracleAddress) {
        owner = msg.sender;
        oracleAddress = initialOracleAddress;
        chatRunsCount = 0;
    }
  
    modifier onlyOwner() {
        require(msg.sender == owner, "Caller is not owner");
        _;
    }
  
    modifier onlyOracle() {
        require(msg.sender == oracleAddress, "Caller is not oracle");
        _;
    }
  
    function setOracleAddress(address newOracleAddress) public onlyOwner {
        oracleAddress = newOracleAddress;
        emit OracleAddressUpdated(newOracleAddress);
    }
  
    function startChat(string memory message) public returns (uint i) {
        ChatRun storage run = chatRuns[chatRunsCount];
      
        run.owner = msg.sender;
        Message memory newMessage;
        newMessage.content = message;
        newMessage.role = "user";
        run.messages.push(newMessage);
        run.messagesCount = 1;
      
        uint currentId = chatRunsCount;
        chatRunsCount = chatRunsCount + 1;
      
        IOracle(oracleAddress).createLlmCall(currentId);
        emit ChatCreated(msg.sender, currentId);
      
        return currentId;
    }
  
    function onOracleLlmResponse(
        uint runId,
        string memory response,
        string memory errorMessage
    ) public onlyOracle {
        ChatRun storage run = chatRuns[runId];
        require(
            keccak256(abi.encodePacked(run.messages[run.messagesCount - 1].role)) == keccak256(abi.encodePacked("user")),
            "No message to respond to"
        );
      
        Message memory newMessage;
        newMessage.content = response;
        newMessage.role = "assistant";
        run.messages.push(newMessage);
        run.messagesCount++;
    }
  
    function addMessage(string memory message, uint runId) public {
        ChatRun storage run = chatRuns[runId];
        require(
            keccak256(abi.encodePacked(run.messages[run.messagesCount - 1].role)) == keccak256(abi.encodePacked("assistant")),
            "No response to previous message"
        );
        require(
            run.owner == msg.sender, "Only chat owner can add messages"
        );
      
        Message memory newMessage;
        newMessage.content = message;
        newMessage.role = "user";
        run.messages.push(newMessage);
        run.messagesCount++;
        IOracle(oracleAddress).createLlmCall(runId);
    }
  
    function getMessageHistoryContents(uint chatId) public view returns (string[] memory) {
        string[] memory messages = new string[](chatRuns[chatId].messages.length);
        for (uint i = 0; i < chatRuns[chatId].messages.length; i++) {
            messages[i] = chatRuns[chatId].messages[i].content;
        }
        return messages;
    }
  
    function getMessageHistoryRoles(uint chatId) public view returns (string[] memory) {
        string[] memory roles = new string[](chatRuns[chatId].messages.length);
        for (uint i = 0; i < chatRuns[chatId].messages.length; i++) {
            roles[i] = chatRuns[chatId].messages[i].role;
        }
        return roles;
    }
}

Agent orchestration & tools

AI agents run in loops, which allow them to reason over the goals, break them down into tasks, and decide when to use tools such as a web browser, code interpreter, terminal, etc. All of this is necessary to enable autonomy and highly capable AI agents. We are bringing agent tools on-chain through our teeML implementation. It can run any arbitrary tool in an off-chain oracle, executed in a trusted execution environment (TEE). teeML provides attestations that anybody can verify off-chain. Furthermore, as teeML is open-sourced, developers can freely fork and deploy their own versions.

Encrypted user data

AI is a highly personalized technology that is customized based on data. Thus, we aim to bring user data on-chain in an encrypted manner so the AI apps/agents that developers build can be truly unique and guarantee privacy for end-users. L1 is as valuable as much as the heterogenous state it provides; this is especially true in the case where data is one of the core primitives of enabling the use cases, as is the case with AI. Besides user data, AI agents/apps also need to have their state as embeddings for long-term memory. Enabling encrypted user data is still an active research area and part of our longer-term roadmap. Initially, we target use cases where all the data is public, akin to existing L1s.

Verifiable AI inference

Problem

How do you bring model inference on-chain? There are three major problems to overcome.

  1. LLMs and similar models require a huge amount of FLOPS. Even though on the edge inference is becoming feasible thanks to quantization and hardware improvements, the largest models still need multi-GPU setups for inference.

  2. The latency of LLMs significantly exceeds block time (e.g. 12 seconds on Ethereum). One call to GPT-4 can easily take 20 seconds, and an agent running tens of iterations can take total latency to several minutes.

  3. LLMs are not always deterministic, and even if deterministic LLM inference is possible (with some loss in speed), it is impossible to get consensus on calling external APIs.

The solution space for making AI inference verifiable on-chain is wide and all of them come with trade-offs. Let’s break down the existing solutions.

zkML

Zero-knowledge machine learning (zkML) enables the ML models to be converted into ZK circuits, and for every inference, a proof is generated, which can be later verified on-chain. It provides the highest security guarantees as it relies on cryptographic security, but the downside is a 180x compute overhead. For example, if it takes 0,5 seconds to run a random forest, then it takes 90 seconds to generate the proof. In the case of larger models such as LLMs, it’s simply unfeasible because the proving time grows exponentially with the size of the model parameters. This means that both latency and cost-wise, zkML is not yet ready for its prime time. In the long term, we believe that zkML will have order-of-magnitude improvements and become efficient as zkML libraries are improved and specialized hardware is developed. Vitalik Buterin has also written about this.

opML

Optimistic machine learning (opML) enables off-chain inference using the optimistic approach with an on-chain dispute engine, akin to optimistic rollups on L2s. opML complies ML models to a specialized VM instruction set, which is then run in its VM. The input and output are posted on chain, which another party can then challenge. The challenging party provides a pinpoint where, in the computation graph of the neural net, the result is false. This pinpoint is then sent to an on-chain smart contract, which provides arbitration. Thus, opML is much faster than zkML, but there is always a delay for the challenge period; the models must be deterministic, and the input data is always public.

opML + zkML

What if you could combine opML and zkML? Ora recently introduced optimistic privacy-preserving AI (opp/ai), which aims to combine the privacy-preserving capabilities of zkML with the computational efficiency of opML. In a nutshell, only the first few layers of the neural net are run in zkML, and the rest of the model is run in opML. This provides model privacy while reducing ZK overhead and, theoretically, data privacy if the ZK proof is generated on the client side. A simple MNIST CNN achieves an 80% reduction in proof generation compared to running the whole model through a zkML circuit. But even in this case, the proof generation takes 36 seconds on a beefy VM with 512GB of RAM. In the case of Stable Diffusion, the most popular open-source image generation model, it would still take 19-180TB of RAM and 2-17 days to generate proofs for only the attention layers of the model. Combining opML and zkML is a promising avenue, but an order-of-magnitude improvement in cost and latency is still needed. Also, it can only work with deterministic models, which inhibits its adoption because GPT-4, Mixtral, etc., are all MoE (mixture of expert) models, which makes them non-deterministic.

FHE

Fully homomorphic execution enables data to stay private throughout the computation, making it desirable for AI use cases. However, the current overhead of FHE compute is many orders of magnitude above what is acceptable for AI use cases.

Trusted execution environment

All of the above solutions are developing fast but still infeasible today for frontier LLMs: the overhead of native execution, zkML and FHE is too large and opML can only run deterministic models. However, there is an alternative that enables building on-chain AI already today: running an async oracle in a TEE. This solves all three problems mentioned above:

  1. The large compute requirement is solved by moving the compute off-chain and into a trusted third party service;

  2. Latency is solved by running the oracle async (the request and response are in different blocks), similar to ChainLink oracles;

  3. Consensus is not required, because the verifiability is provided by the trusted enclave, which produces attestations of every execution on-chain. Only trust in the enclave hardware manufacturer is required.

As the first step, we aim to bring verifiable inference on-chain, which is low latency and cost-competitive. The reasoning is that initially, we don’t expect that most AI apps or agents require high computation integrity, which would, for example, be necessary in the case of enabling DeFi with ML. Instead, low latency, throughput, and cost are much more critical. To achieve those constraints, we developed teeML: Trusted Execution Environment for Machine Learning Inference. It enables the querying of any of the existing LLM APIs, both for open and closed-source models, in a low latency, low cost, and verifiable manner. teeML node is run off-chain along with an on-chain oracle contract. Developers only need to call out the oracle contract, and a result is returned to them with a call-back. The teeML generates an attestation of the correct code execution, which anybody can verify off-chain.

teeML execution flow

Trusted execution environments (TEEs) have also been recently proposed to solve Ethereum’s MEV by producing ETH blocks in TEEs, which show its promise in blockchain applications. teeML’s strengths are low latency, cost, and high throughput. On the downside, enclaves can be potentially exploited by a well funded adversary and attacks along with mitigation techniques have been published. As well, enclave execution can’t be fully verified on-chain, thus it’s not a fully trustless system. We at Galadriel run the teeML oracle, which means it’s akin to a centralized sequencer in the case of L2s. For a more trustless setup, developers can also fork or run their own teeML oracle if they choose to. We also aim to incentivize more oracles to provide on-chain inference.


Validator level inference

teeML is the first step towards progressive decentralization and as part of our roadmap, we are planning to develop a fully verifiable AI inference on the node level that can run any open-source model. This meets the needs of securing high TVL use cases as it is trustless and is part of the network consensus. We will be writing more about this in the future.

Low-latency, low-cost, parallel execution EVM

Imagine using AI applications as an end-user and waiting for 10+ seconds. Most users will churn on this because it’s simply bad UX. As many AI apps/agents are end-user-focused, they have to be able to respond with low latency. Furthermore, when agents call out each other, i.e., imagine an agent swarm with 100 agents combined to achieve a goal, it can’t have an order of magnitude overhead from cost or time-wise because otherwise, it can’t compete with centralized solutions. Blockchains introduce cost and latency overhead, and we aim to reduce that overhead for AI agents/apps as much as possible. Thus, we are building Galadriel on a parallel execution EVM stack, specifically off Cosmos-SDK and Sei v2.

Safety & Governance

We believe prioritizing AI safety and governance is not only a morally right thing to do stemming our mission but we also believe that in the upcoming years market will put a premium on it. Regulation will likely try to enforce it and society will not tolerate technology that has a risk to human lives - just see how market reacted to Boeing’s two fatal incidents in 2020.

Challenges

Besides centralized AI, building on-chain AI has it’s own set of challenges:

  • In a fork, any safety system can be modified or removed, making the fork more dangerous;

  • Since all safety mechanisms are public, they can be adversarially optimized against without any public trace (e.g. developing adversarial examples);

  • Standard anti-abuse mechanisms like staking and slashing are harder to use because the abusive behaviour is harder to define objectively (what constitutes a malicious LLM reply?);

  • It is hard to coordinate between the mostly pseudonymous actors in the community;

  • How to shut down agents that fork themselves and become malicious in a decentralized setting.

Core theses

Before we delve into the solution space, we’d like to bring out some of our core theses which impact the design space of safety & governance:

  • Mustafa Suleyman, co-founder of Deepmind has stated that technology always diffuses throughout the world and the same will happen with AGI. There’s just a time delay between the parties who gain access first and who later. This can also be observed with LLMs where it took open-source just over a year to catch up to GPT-4 with the release of Mixtral. Thus we operate on the premise that AGI will eventually diffuse and become widely available;

  • In the 1980s, PCs became useful as standalone devices, but their true value was unlocked by networking them together. Likewise, AI models today are standalone, but their true value will unlock through a network of AIs. AI will gravitate towards the network with the highest utility and access to FLOPS and data. See more in the agent swarms section;

  • AI risks have three distinct phases. Phase one is about humans using AI for malicious purposes - hacking, disinformation, propagating biases, etc. After that, in phase two, once AI gains autonomy, the risk shifts from humans to AI itself becoming the threat model. Thirdly, once superintelligence is achieved it’s a black hole and the outcome is unknown;

  • AI will become destructive because history has shown that it is in human nature that if a technology can be weaponized, then it will be. Meaning, AI itself might be harmless but humans themselves will use it for malicious purposes.

Solutions

The L1 design should incorporate game theory for safety and minimize harmful use-cases. There is no single silver bullet for aligning AI today but we see many promising avenues which we aim to pursue:

  • L1 governing laws could be based upon United Nations Declaration of Human Rights in order not to harm humans. This is inspired by Antrophic’s research in constitutional AI;

  • The L1 game theory should be designed to protect against external adversaries i.e. attacks against the chain itself and also internal adversaries i.e. destructive AI agents which attack the external world;

  • Network could have a mechanism to vote to shut off malicious agents. The voting could also be done potentially by other agents which provide a proof of malicious behaviour and collect a reward for it;

  • Compared to black boxes such as OpenAI, etc. servers, Web3 rails provide transparency. This is crucial in building safe and credible oversight once developers start building and deploying millions of AI agents. In case anything starts going haywire it can be acted upon quickly but the same can’t be said about the servers of OpenAI;

  • Governance could have a kill switch. Network participants could have a method to shut down the network if the network becomes destructive for the society and if they choose to vote upon it. A natural kill switch has also been proposed by Vitalik Buterin;

  • The most cyber-secure systems (BTC, ETH, Linux, etc.) in the world have 4 core properties: they are open-source, they are game theoretic & driven by incentives, they operate in a constant adversarial setting and they leverage wisdom of the crowd. Galadriel’s design is based upon those four core properties.

In the last ten years, applied cryptography has blossomed - everything from FHE to ZK, which was thought to be much farther away. This is mainly due to Ethereum and other L1 ecosystems, which have incentives to invent better crypto primitives and deploy them into production. Alike, Galadriel could become the world’s biggest applied AI safety network, where crypto incentives drive researchers and engineers to align AI. We believe deploying AI at scale on Web3 rails which is constantly battle-tested and improved by thousands of open-source participants will lead to the most secure rails for AI. We will hire and be the first Web3 network with an AI safety team. Its goal is to develop a robust methodology for measuring and progressing AI’s alignment on Galadriel.

Comparison with existing decentralized AI projects

Today, AIxWeb3 is still extremely small, and we believe a rising tide raises all boats. AI works off three core primitives: FLOPS, data, and human coordination—all of those core components have to be built up with crypto-native incentives. We are only scratching the surface, and the infrastructure buildout has only started, which is necessary to train, infer, and deploy AI at scale on Web3 rails. Here’s how Galadriel compares to the existing Web3xAI projects.

We aim to collaborate with many and believe all of the projects are necessary for the Web3xAI infrastructure’s buildout. Compared to others, Galadriel is the first L1 for AI that enables AI-native applications/agents the same way Ethereum enabled dApps.

Use cases

We expect many of the initial use cases to be skeuomorphic as developers start experimenting in this new design space where they can build Web3xAI native apps for the first time—our goal is to enable 1000 flowers to blossom. As it was hard for Vitalik Buterin to predict all of the use cases in 2014 with Ethereum, we are conscious that most use cases today have yet to be invented. Yet, here are some of the potential ones we foresee.

Agent swarms

It’s well-researched that multi-agent systems in complex tasks that require a set of fundamental abilities such as reasoning, multi-modality handling, web browsing, and general tool use outperform frontier LLMs on their own. On GAIA: A Benchmark for General AI Assistants by Meta, Hugging Face, etc., a multi-agent system performs almost 5x better than GPT4-Turbo, as seen on the GAIA Leaderboard.

Thus, Agent swarms can accomplish far more complex tasks when networked together. Swarm intelligence can also be observed in nature, such as in the case of bees, ants, and even humans. Large groups of relatively simple organisms can compute the solutions to complex problems by collectively organizing their behavior in such a way that the actions of one individual do not make much sense, but when put in the context of thousands of other individuals all following the same rules, complex computation takes place. Swarm intelligence follows a given set of steps in social animals, which shows the mechanisms at play in this type of meta-cognition:

  1. Information sharing: the information that each member obtains in its search needs to be shared with the whole swarm so that a decision can be made.

  2. Evaluation: a method for evaluating the information that has been shared must exit so that the swarm can make meaningful decisions. In the case of the honeybee, this method is built in how bees choose which dancing bees to follow. If there are several dancing bees, which one should each of the other bees follow into the new home? The answer is simple: the bee whose dance is more enthusiastic or convincing. Different bees seem to have various ‘opinions’ as to which of their mates makes a better ‘speech,’ giving rise to a sort of ‘elections’ in which there is no absolute consensus.

  3. Consensus building: In the case of bees, when a consensus has not yet been reached, and the difference in the ‘voting outcome’ is not that big (55% versus 45%), a ‘debate’ takes place. The bees start making loud noises and bumping against bees who have a different view from their own. In this manner, some of the bees that receive this type of treatment, as it is uncomfortable for them, change their mind about who to follow in favor of the view of the bee that is ‘confronting’ it. By this method, the swarm avoids deadlock between competing alternatives and ensures that the option that is to be chosen by the majority of the bees is indeed the right one.

  4. Committing on the vote: Now that the colony has made a clear decision by majority vote, the whole swarm follows the winning bee to their new hive.

Today, multi-agent systems follow similar mechanics, and Web3 provides them native rails to enable swarm intelligence - like the DeFi Lego blocks moment where simple core primitives have enabled developers to build a complex interlocking financial system. In the long-term, there’s also a fundamental question of whether AGI will be reached through a singleton, i.e., a single model, or through sorts of swarm intelligence that can be observed through evolution. The existing research seems to point to the latter. We believe Galadriel can significantly contribute to finding that out as we plan to scale on-chain swarm intelligence to millions of interlocking and interacting agents. This, in turn, changes the L1 from a deterministic state machine to more of a living organism driven by adaption - the hallmark of intelligence.

Culture/Social

We loved when NFTs hit the mainstream because they brought culture on-chain and showed that Web3 can be more than just serious financial applications. AI opens up a new experimentation canvas for artists and developers alike. For example, imagine generative NFTs that are constantly changing and evolving. Or, imagine your favorite meme coin becoming alive and having a much deeper connection with it - might it be dogwifhat, Pepe, or Doge.

On-chain Midjourney: https://minter.galadriel.com/
We also imagine use cases where existing centralized products will be decentralized, which can lead to better products with the right incentives. For example, in the same way, Farcaster is building a user-owned social media platform, we envision developers building user-owned ChatGPT, Midjourney, Character.ai, etc.

On-chain ChatGPT:https://chatgpt.galadriel.com/

AI Gaming & Simulations

Imagine a world where AI agents live and interact with each other in a virtual environment, learning and evolving just as humans do. Imagine playing this game where the world is ever-evolving. Google created this kind of Sims-like game in a small scale and we envision developers to scale these sorts of simulations up to millions of agents inhibiting complex behaviours.

We also see games where AI agents compete against each other, and users can bet on winners, like hamsters.gg, or users can compete themselves with on-chain AI to win prizes. As an example, we built VitAIlik: an RPG game where users can battle with an on-chain AI version of Vitalik. Every game is new, as AI generates the lore and gameplay dynamically. Much more of future gaming will be dynamically generated by AI based on the users’ preferences, and we see many of them coming on-chain.

On-chain VitAIlik game:https://vitailik.galadriel.com/

Finance & Settlement Layer for AI

We believe, in the long term, AI will drive GDP growth in the world most through its ability to conduct business between companies on a global scale. Effectively, companies are going from operating at a human speed to machine speed. It’s like the Internet made information move at the speed of light, but now it’s for whole companies. For this, a native payment and settlement layer is necessary where AI agents can exchange money in a trustless manner for the exchanged services.

Secondly, imagine you are part of a DAO but the managing of the treasury is not trustless. We’ve seen the exact problem in many DAOs and this could for example be solved by an AI DAO.

An AI DAO where the treasury is managed by AI agents

Furthermore, we see endless opportunities where AI can improve today’s Web3 products or create new categories by giving AI money and thus increasing its utility:

  • Automated yield farming;

  • Agent-based prediction markets;

  • On-chain insurance run by AI;

  • Hedge funds managed by AI;

Let 1000 flowers blossom

As Vitalik Buterin couldn’t foresee many of today’s use cases, such as NFTs, we imagine most of the use cases are yet to be dreamed of and invented. That’s why our goal is to enable developers to play with this new design space and experiment easily so that 1000 flowers can blossom.

Roadmap

Move fast, ship to users

We as a team have gone with multiple AI & Web3 startups from 0 → 1 and afterwards helped to scale to unicorns. Thus, we believe in the ethos of moving fast and iterating against users vs trying to come up with a perfect solution without grounding in reality. Thus, we err on the side of shipping early which also means if you are developing on top of Galadriel during devnet expect things to evolve and change rapidly.

Research

Besides, the core product there’s fundamental research to be conducted on the following areas: enabling privacy for user data, scalability in verified AI inference, bringing embeddings on chain, AI safety research and more.

Devnet

Galadriel’s Devnet is out now. During devnet developers can build working on-chain AI agents and applications. The focus is on developing the core technology, quickly iterating the product along with user needs and progressively decentralizing. We expect to make many big architectural changes during the devnet as the full chain spec is yet to be implemented.

Testnet

After the validation of the design and architecture during devnet, we aim to lock the chain spec and release a testnet. Biggest upgrade will be in bringing AI inference on the validator level so the network can achieve highest security and verifiability guarantees. It will act as the final testing ground before mainnet together with a security audit. We seek to reward and incentivise the early adopters and community members.

Mainnet

Following the successful iterations and invaluable feedback garnered through our testnet phase, we will launch Galadriel’s mainnet. In line with our tradition of rewarding early adopters and community members, the mainnet launch will distribute incentives and introduce new reward mechanisms.

Vision of Ethereum for AI

Galadriel’s mission is to build safe, citizen owned AI by building ETH for AI. Our long-term vision can be unpacked as:

  1. Build the first L1 for AI which acts as the settlement layer for AI, driving real GDP growth.

  2. Provide direct ownership and democratic governance over AI.

  3. Become the most secure rails for AI deployment and largest applied safety network where crypto-incentives drive humanity to align AI.

What is the future of AI? Will it be closed, corporate-owned, or safe and user-owned? Become a Galadrian and join the mission in making the latter happen.

The Creation of a New Species

Today, we are on the cusp of inventing a new species on this planet—one that runs on silicon and, in evolutionary terms, is evolving orders of magnitudes faster than anything else this planet has seen. This can be observed in the 1000x AI compute increase in the past eight years on the left image as well, measured in the model’s capability increase as seen on the right image.

Five years ago, before GPT-3, experts predicted AGI to be 80 years away, and today, the median prediction is 2030. This shows that once AI starts accelerating as it is now, predicting the outcomes is extremely hard because of its exponential nature.

Also, the people at the forefront of AI predict humanity will reach AGI by 2030 or possibly sooner than that, including Sam Altman, Demis Hassabis, Jensen Huang, Dario Amodei, and Elon Musk.

As a team, we’ve been in the AI industry since 2018, and all of this progress is breathtaking and extremely exciting. However, we are increasingly worried because today’s approach to AI safety, ownership, and governance is fundamentally broken. Let’s see why.

“Unfortunately, humanity’s future is in the hands of…”

In 2018, Elon Musk wrote the following letter to the founders of OpenAI.

“Unfortunately, humanity’s future is in the hands of Larry Page (the redacted part).” Elon founded OpenAI as a counterbalance to Google, but unfortunately, the situation is now even worse because of the acceleration risk—a racing dynamic between companies leading to a decline in safety standards, the diffusion of bad norms, and accelerated AI timelines stated by OpenAI. AI, as of today, is controlled, governed, and owned only by a few giant monopolies, including OpenAI, Microsoft, Google, Meta, and a few closed-sourced AI labs. Would you trust your or your family’s lives in the hands of Sam Altman? The man who promises to bring the world a safe AI when he can’t even govern his own company, as shown by the board hiatus. No one is admitting the fact, but today, the fate of 8 billion people is in the hands of less than 50 people who control AI - boards of the aforementioned companies.

As a team, we have worked in cybersecurity, AI safety, and Web3 since 2015,. Throughout our years in cybersecurity, we’ve learned that security through obscurity never works - history has repeatedly shown that. For example, a Chinese spy was recently caught stealing AI secrets from Google. We expect AI risks to have multiple phases. Phase one is about humans using AI for malicious purposes - hacking, disinformation, propagating biases, etc. After that, in phase two, once AI gains autonomy, the risk shifts from humans to AI itself becoming the threat model. Autonomy enables everything from “malware with brains,” which could aim to infect the whole Internet, as there have been examples in the case of Mydoom, which at its peak sent 25% of all emails on the Internet or Zeus, which breached 88% of all Fortune 500 companies. Autonomy will further lead to AI directing its evolution as it could start forking itself through retraining with new data that it could detect to be missing from achieving its goal. Furthermore, there are long-term risks with superintelligence as the control problem is yet to be solved. The reason why we used to work in cybersecurity and NATO is because cybersecurity has never been for us about protecting technical systems but protecting human lives - when critical systems in hospitals or electrical grids are taken down, lives are on the line. Considering the previous, we believe the AI oligopoly’s existing approach to safety is fundamentally flawed, and not learning from history will most likely lead to disastrous results for society.

Besides the lack of safety, another glaring problem is the lack of ownership and governance. In the ’90s, there was a strong push for corporate networks by the likes of Microsoft, AOL, etc., meaning that the Internet would be corporate-owned, taxed, and controlled - imagine the Internet being like the App Store. Today, AI is akin to the 90’s corporate intranets where they live on their permissioned and closed-off islands. AI will be the biggest wealth creator throughout human history, but suppose it is solely owned and governed by a handful of existing Web2 monopolies as of today. In that case, the most likely outcome is extreme wealth disparity because of their extractive economic model. Imagine in 5 years OpenAI’s frontier model being able to conduct 50% of the workforce. OpenAI is predicting outcomes as such in its recent research: GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models. In the next coming years, AI will introduce a large scale workforce dislodge and seen in the chart below.

Exposure to LLMs by industry, from the paper GPTs are GPTs

Furthermore, no right-minded entrepreneur will build today on top of Facebook because it shut down all of the companies on its platform, most famously Zynga - almost all of the large Web2 platforms followed the same model with high take rates and anti-competitive behaviour. In a world where AI is closed and owned by a few oligopolies, there is no opportunity for people to own what they build, receive fair compensation for it, or have a say in its progress.

Safe, citizen owned AI

Until January 3, 2009, there was no secure system in the world from the cybersecurity perspective - not in Google, Meta, or even the NSA itself, which has been hacked multiple times. This changed with the invention of Bitcoin and later Ethereum - the proof is in the pudding as Bitcoin secures $1.3T+ and Ethereum $478B+ as of 13.3.2024. They are the world’s biggest bounties and have withstood penetration tests from individual black hat hackers to nation-states. Humanity has invented an impenetrable system for the first time in computer science history.

Secondly, blockchains can be owned by everyone, which means their ownership scales to tens of millions of users, as in the case of Bitcoin and Ethereum, which leads to a far greater wealth distribution. Contrast this to OpenAI, which latest valuation was $80B, yet none of the value has accrued to the users because it is owned by approximately 600 people, counting the team and investors. We believe, with the right incentives, decentralized AI networks can outcompete today’s closed AI - just like the Internet did for the intranets.

Thirdly, blockchains enable scalable governance. Their governance at times might be messy, as seen by the Bitcoin blocksize wars. Still, it is a far superior governance system where thousands of participants - developers, users, and investors can direct the progress compared to a handful of people. This is crucially important given we are inventing a new species on this planet - everyone who wishes should have an opportunity to have a say in it. Furthermore, contrary to popular belief, the community can effectively govern blockchains to stop illicit activity. Examples range from Bittensor shutting down a subnetwork generating illicit material to Filecoin addressing copyright violations.

Thus, we believe large-scale AI deployment should happen on Web3 rails, which is proven to be the most secure infrastructure to date, enabling user ownership and democratic governance. Our mission is to build a safe, citizen-owned AI.

Ethereum for AI

In 2015, Ethereum showed that the ability to write arbitrary software that can provide trust guarantees turned out to be really powerful, and it became the underpinning of Web3. Today, if you are a developer building AI apps or agents, the only option is to deploy them on a centralized stack. During the last OpenAI Dev Day, many developers felt rug pulled as OpenAI bundled their products into GPTs and an assistant API. All centralized AI providers come with a platform risk for devs because they compete against them as they use their data to build competing apps and train next-frontier models, making dev products irrelevant.

Like Ethereum, which enabled writing smart contracts to build dapps, Galadriel enables developers to build AI apps/agents like smart contracts, which provide trust guarantees and are decentralized. Today, developers understand the value of decentralization, which ensures they have a platform that can’t rug-pull them. This is as valid for building with AI because developers want to be the masters of their destiny. Galadriel’s goal as the first L1 for AI is to become the application layer for AI apps/agents, just like ETH has for d(apps).

Building an L1 for AI is no easy feat because, among other reasons, the AI and Web3 compute paradigms are worlds apart. AI favors centralization as it’s scaled on data centers with high-bandwidth interconnects, whereas Web3 is built on decentralization and verifiability, which introduces a large compute overhead. Furthermore, it seems most are trying to merge these two worlds just because it sounds plausibly good without solving any real problem. There’s a strong need to reason from the first principles to avoid that trap. We initially started building Galadriel as an L2 but realized that vertical integration is necessary to enable fully decentralized & autonomous AI agents/apps. Vitalik initially thought of building smart contracts on Bitcoin but was said no by the core developers. This turned out to be an exceptionally good outcome because Ethereum also had to be vertically integrated to enable smart contracts. Purpose-built tools enable purpose-built use cases, and AI is one of them. Galadriel’s L1 stack is the following:

Now, let’s break it down.

Autonomy

The holy grail of AI will become autonomy. Just like humans evolved a higher level of autonomy than animals, AI’s next step in its evolution is toward more autonomy. The AI labs are already working on that through agentic LLMs (1, 2, 3). This means that users can describe higher-level tasks/goals to AI, and it puts together a plan and executes it step-by-step. In essence, it means we will interact with AI in the next few years, just as with a team member: they will become our software engineers, marketers, accountants, lawyers, etc. Devin, the first AI software engineer, is an excellent example of that. Humans allocate resources based on the complexity and reward of the goals, and the next frontier models will have a dynamic inference budget allocated depending on the complexity of the task. For example, given a mathematical formula to prove, the model could run for hours to find the proof; meanwhile, generating a blog post might take seconds.

Yet, AI can’t achieve full autonomy in the centralized servers of OpenAI, etc, because of the platform risk. We believe that once AI achieves autonomous capabilities, it will trend towards rails, which provide trust guarantees for autonomy. Today, no blockchain natively supports autonomy. We aim to deliver node-level capability for full autonomy, meaning agents can self-execute, fork, interact, trade, etc, without the risk of relying on centralized third parties.

DevExp in Solidity

EVM has the most significant mind share among developers and the biggest network effects. Thus, we aim to provide developers with a frictionless and familiar development experience without having to learn a totally new stack from scratch. This is crucial to letting 1000 flowers blossom and enabling a wide variety of experimentation.

Let’s see an example implementation of an on-chain ChatGPT:

// SPDX-License-Identifier: UNLICENSED
pragma solidity ^0.8.9;

interface IOracle {
    function createLlmCall(
        uint promptId
    ) external returns (uint);
}

contract ChatGpt {
    struct Message {
        string role;
        string content;
    }
  
    struct ChatRun {
        address owner;
        Message[] messages;
        uint messagesCount;
    }
  
    mapping(uint => ChatRun) public chatRuns;
    uint private chatRunsCount;
  
    event ChatCreated(address indexed owner, uint indexed chatId);
  
    address private owner;
    address public oracleAddress;
  
    event OracleAddressUpdated(address indexed newOracleAddress);
  
    constructor(address initialOracleAddress) {
        owner = msg.sender;
        oracleAddress = initialOracleAddress;
        chatRunsCount = 0;
    }
  
    modifier onlyOwner() {
        require(msg.sender == owner, "Caller is not owner");
        _;
    }
  
    modifier onlyOracle() {
        require(msg.sender == oracleAddress, "Caller is not oracle");
        _;
    }
  
    function setOracleAddress(address newOracleAddress) public onlyOwner {
        oracleAddress = newOracleAddress;
        emit OracleAddressUpdated(newOracleAddress);
    }
  
    function startChat(string memory message) public returns (uint i) {
        ChatRun storage run = chatRuns[chatRunsCount];
      
        run.owner = msg.sender;
        Message memory newMessage;
        newMessage.content = message;
        newMessage.role = "user";
        run.messages.push(newMessage);
        run.messagesCount = 1;
      
        uint currentId = chatRunsCount;
        chatRunsCount = chatRunsCount + 1;
      
        IOracle(oracleAddress).createLlmCall(currentId);
        emit ChatCreated(msg.sender, currentId);
      
        return currentId;
    }
  
    function onOracleLlmResponse(
        uint runId,
        string memory response,
        string memory errorMessage
    ) public onlyOracle {
        ChatRun storage run = chatRuns[runId];
        require(
            keccak256(abi.encodePacked(run.messages[run.messagesCount - 1].role)) == keccak256(abi.encodePacked("user")),
            "No message to respond to"
        );
      
        Message memory newMessage;
        newMessage.content = response;
        newMessage.role = "assistant";
        run.messages.push(newMessage);
        run.messagesCount++;
    }
  
    function addMessage(string memory message, uint runId) public {
        ChatRun storage run = chatRuns[runId];
        require(
            keccak256(abi.encodePacked(run.messages[run.messagesCount - 1].role)) == keccak256(abi.encodePacked("assistant")),
            "No response to previous message"
        );
        require(
            run.owner == msg.sender, "Only chat owner can add messages"
        );
      
        Message memory newMessage;
        newMessage.content = message;
        newMessage.role = "user";
        run.messages.push(newMessage);
        run.messagesCount++;
        IOracle(oracleAddress).createLlmCall(runId);
    }
  
    function getMessageHistoryContents(uint chatId) public view returns (string[] memory) {
        string[] memory messages = new string[](chatRuns[chatId].messages.length);
        for (uint i = 0; i < chatRuns[chatId].messages.length; i++) {
            messages[i] = chatRuns[chatId].messages[i].content;
        }
        return messages;
    }
  
    function getMessageHistoryRoles(uint chatId) public view returns (string[] memory) {
        string[] memory roles = new string[](chatRuns[chatId].messages.length);
        for (uint i = 0; i < chatRuns[chatId].messages.length; i++) {
            roles[i] = chatRuns[chatId].messages[i].role;
        }
        return roles;
    }
}

Agent orchestration & tools

AI agents run in loops, which allow them to reason over the goals, break them down into tasks, and decide when to use tools such as a web browser, code interpreter, terminal, etc. All of this is necessary to enable autonomy and highly capable AI agents. We are bringing agent tools on-chain through our teeML implementation. It can run any arbitrary tool in an off-chain oracle, executed in a trusted execution environment (TEE). teeML provides attestations that anybody can verify off-chain. Furthermore, as teeML is open-sourced, developers can freely fork and deploy their own versions.

Encrypted user data

AI is a highly personalized technology that is customized based on data. Thus, we aim to bring user data on-chain in an encrypted manner so the AI apps/agents that developers build can be truly unique and guarantee privacy for end-users. L1 is as valuable as much as the heterogenous state it provides; this is especially true in the case where data is one of the core primitives of enabling the use cases, as is the case with AI. Besides user data, AI agents/apps also need to have their state as embeddings for long-term memory. Enabling encrypted user data is still an active research area and part of our longer-term roadmap. Initially, we target use cases where all the data is public, akin to existing L1s.

Verifiable AI inference

Problem

How do you bring model inference on-chain? There are three major problems to overcome.

  1. LLMs and similar models require a huge amount of FLOPS. Even though on the edge inference is becoming feasible thanks to quantization and hardware improvements, the largest models still need multi-GPU setups for inference.

  2. The latency of LLMs significantly exceeds block time (e.g. 12 seconds on Ethereum). One call to GPT-4 can easily take 20 seconds, and an agent running tens of iterations can take total latency to several minutes.

  3. LLMs are not always deterministic, and even if deterministic LLM inference is possible (with some loss in speed), it is impossible to get consensus on calling external APIs.

The solution space for making AI inference verifiable on-chain is wide and all of them come with trade-offs. Let’s break down the existing solutions.

zkML

Zero-knowledge machine learning (zkML) enables the ML models to be converted into ZK circuits, and for every inference, a proof is generated, which can be later verified on-chain. It provides the highest security guarantees as it relies on cryptographic security, but the downside is a 180x compute overhead. For example, if it takes 0,5 seconds to run a random forest, then it takes 90 seconds to generate the proof. In the case of larger models such as LLMs, it’s simply unfeasible because the proving time grows exponentially with the size of the model parameters. This means that both latency and cost-wise, zkML is not yet ready for its prime time. In the long term, we believe that zkML will have order-of-magnitude improvements and become efficient as zkML libraries are improved and specialized hardware is developed. Vitalik Buterin has also written about this.

opML

Optimistic machine learning (opML) enables off-chain inference using the optimistic approach with an on-chain dispute engine, akin to optimistic rollups on L2s. opML complies ML models to a specialized VM instruction set, which is then run in its VM. The input and output are posted on chain, which another party can then challenge. The challenging party provides a pinpoint where, in the computation graph of the neural net, the result is false. This pinpoint is then sent to an on-chain smart contract, which provides arbitration. Thus, opML is much faster than zkML, but there is always a delay for the challenge period; the models must be deterministic, and the input data is always public.

opML + zkML

What if you could combine opML and zkML? Ora recently introduced optimistic privacy-preserving AI (opp/ai), which aims to combine the privacy-preserving capabilities of zkML with the computational efficiency of opML. In a nutshell, only the first few layers of the neural net are run in zkML, and the rest of the model is run in opML. This provides model privacy while reducing ZK overhead and, theoretically, data privacy if the ZK proof is generated on the client side. A simple MNIST CNN achieves an 80% reduction in proof generation compared to running the whole model through a zkML circuit. But even in this case, the proof generation takes 36 seconds on a beefy VM with 512GB of RAM. In the case of Stable Diffusion, the most popular open-source image generation model, it would still take 19-180TB of RAM and 2-17 days to generate proofs for only the attention layers of the model. Combining opML and zkML is a promising avenue, but an order-of-magnitude improvement in cost and latency is still needed. Also, it can only work with deterministic models, which inhibits its adoption because GPT-4, Mixtral, etc., are all MoE (mixture of expert) models, which makes them non-deterministic.

FHE

Fully homomorphic execution enables data to stay private throughout the computation, making it desirable for AI use cases. However, the current overhead of FHE compute is many orders of magnitude above what is acceptable for AI use cases.

Trusted execution environment

All of the above solutions are developing fast but still infeasible today for frontier LLMs: the overhead of native execution, zkML and FHE is too large and opML can only run deterministic models. However, there is an alternative that enables building on-chain AI already today: running an async oracle in a TEE. This solves all three problems mentioned above:

  1. The large compute requirement is solved by moving the compute off-chain and into a trusted third party service;

  2. Latency is solved by running the oracle async (the request and response are in different blocks), similar to ChainLink oracles;

  3. Consensus is not required, because the verifiability is provided by the trusted enclave, which produces attestations of every execution on-chain. Only trust in the enclave hardware manufacturer is required.

As the first step, we aim to bring verifiable inference on-chain, which is low latency and cost-competitive. The reasoning is that initially, we don’t expect that most AI apps or agents require high computation integrity, which would, for example, be necessary in the case of enabling DeFi with ML. Instead, low latency, throughput, and cost are much more critical. To achieve those constraints, we developed teeML: Trusted Execution Environment for Machine Learning Inference. It enables the querying of any of the existing LLM APIs, both for open and closed-source models, in a low latency, low cost, and verifiable manner. teeML node is run off-chain along with an on-chain oracle contract. Developers only need to call out the oracle contract, and a result is returned to them with a call-back. The teeML generates an attestation of the correct code execution, which anybody can verify off-chain.

teeML execution flow

Trusted execution environments (TEEs) have also been recently proposed to solve Ethereum’s MEV by producing ETH blocks in TEEs, which show its promise in blockchain applications. teeML’s strengths are low latency, cost, and high throughput. On the downside, enclaves can be potentially exploited by a well funded adversary and attacks along with mitigation techniques have been published. As well, enclave execution can’t be fully verified on-chain, thus it’s not a fully trustless system. We at Galadriel run the teeML oracle, which means it’s akin to a centralized sequencer in the case of L2s. For a more trustless setup, developers can also fork or run their own teeML oracle if they choose to. We also aim to incentivize more oracles to provide on-chain inference.


Validator level inference

teeML is the first step towards progressive decentralization and as part of our roadmap, we are planning to develop a fully verifiable AI inference on the node level that can run any open-source model. This meets the needs of securing high TVL use cases as it is trustless and is part of the network consensus. We will be writing more about this in the future.

Low-latency, low-cost, parallel execution EVM

Imagine using AI applications as an end-user and waiting for 10+ seconds. Most users will churn on this because it’s simply bad UX. As many AI apps/agents are end-user-focused, they have to be able to respond with low latency. Furthermore, when agents call out each other, i.e., imagine an agent swarm with 100 agents combined to achieve a goal, it can’t have an order of magnitude overhead from cost or time-wise because otherwise, it can’t compete with centralized solutions. Blockchains introduce cost and latency overhead, and we aim to reduce that overhead for AI agents/apps as much as possible. Thus, we are building Galadriel on a parallel execution EVM stack, specifically off Cosmos-SDK and Sei v2.

Safety & Governance

We believe prioritizing AI safety and governance is not only a morally right thing to do stemming our mission but we also believe that in the upcoming years market will put a premium on it. Regulation will likely try to enforce it and society will not tolerate technology that has a risk to human lives - just see how market reacted to Boeing’s two fatal incidents in 2020.

Challenges

Besides centralized AI, building on-chain AI has it’s own set of challenges:

  • In a fork, any safety system can be modified or removed, making the fork more dangerous;

  • Since all safety mechanisms are public, they can be adversarially optimized against without any public trace (e.g. developing adversarial examples);

  • Standard anti-abuse mechanisms like staking and slashing are harder to use because the abusive behaviour is harder to define objectively (what constitutes a malicious LLM reply?);

  • It is hard to coordinate between the mostly pseudonymous actors in the community;

  • How to shut down agents that fork themselves and become malicious in a decentralized setting.

Core theses

Before we delve into the solution space, we’d like to bring out some of our core theses which impact the design space of safety & governance:

  • Mustafa Suleyman, co-founder of Deepmind has stated that technology always diffuses throughout the world and the same will happen with AGI. There’s just a time delay between the parties who gain access first and who later. This can also be observed with LLMs where it took open-source just over a year to catch up to GPT-4 with the release of Mixtral. Thus we operate on the premise that AGI will eventually diffuse and become widely available;

  • In the 1980s, PCs became useful as standalone devices, but their true value was unlocked by networking them together. Likewise, AI models today are standalone, but their true value will unlock through a network of AIs. AI will gravitate towards the network with the highest utility and access to FLOPS and data. See more in the agent swarms section;

  • AI risks have three distinct phases. Phase one is about humans using AI for malicious purposes - hacking, disinformation, propagating biases, etc. After that, in phase two, once AI gains autonomy, the risk shifts from humans to AI itself becoming the threat model. Thirdly, once superintelligence is achieved it’s a black hole and the outcome is unknown;

  • AI will become destructive because history has shown that it is in human nature that if a technology can be weaponized, then it will be. Meaning, AI itself might be harmless but humans themselves will use it for malicious purposes.

Solutions

The L1 design should incorporate game theory for safety and minimize harmful use-cases. There is no single silver bullet for aligning AI today but we see many promising avenues which we aim to pursue:

  • L1 governing laws could be based upon United Nations Declaration of Human Rights in order not to harm humans. This is inspired by Antrophic’s research in constitutional AI;

  • The L1 game theory should be designed to protect against external adversaries i.e. attacks against the chain itself and also internal adversaries i.e. destructive AI agents which attack the external world;

  • Network could have a mechanism to vote to shut off malicious agents. The voting could also be done potentially by other agents which provide a proof of malicious behaviour and collect a reward for it;

  • Compared to black boxes such as OpenAI, etc. servers, Web3 rails provide transparency. This is crucial in building safe and credible oversight once developers start building and deploying millions of AI agents. In case anything starts going haywire it can be acted upon quickly but the same can’t be said about the servers of OpenAI;

  • Governance could have a kill switch. Network participants could have a method to shut down the network if the network becomes destructive for the society and if they choose to vote upon it. A natural kill switch has also been proposed by Vitalik Buterin;

  • The most cyber-secure systems (BTC, ETH, Linux, etc.) in the world have 4 core properties: they are open-source, they are game theoretic & driven by incentives, they operate in a constant adversarial setting and they leverage wisdom of the crowd. Galadriel’s design is based upon those four core properties.

In the last ten years, applied cryptography has blossomed - everything from FHE to ZK, which was thought to be much farther away. This is mainly due to Ethereum and other L1 ecosystems, which have incentives to invent better crypto primitives and deploy them into production. Alike, Galadriel could become the world’s biggest applied AI safety network, where crypto incentives drive researchers and engineers to align AI. We believe deploying AI at scale on Web3 rails which is constantly battle-tested and improved by thousands of open-source participants will lead to the most secure rails for AI. We will hire and be the first Web3 network with an AI safety team. Its goal is to develop a robust methodology for measuring and progressing AI’s alignment on Galadriel.

Comparison with existing decentralized AI projects

Today, AIxWeb3 is still extremely small, and we believe a rising tide raises all boats. AI works off three core primitives: FLOPS, data, and human coordination—all of those core components have to be built up with crypto-native incentives. We are only scratching the surface, and the infrastructure buildout has only started, which is necessary to train, infer, and deploy AI at scale on Web3 rails. Here’s how Galadriel compares to the existing Web3xAI projects.

We aim to collaborate with many and believe all of the projects are necessary for the Web3xAI infrastructure’s buildout. Compared to others, Galadriel is the first L1 for AI that enables AI-native applications/agents the same way Ethereum enabled dApps.

Use cases

We expect many of the initial use cases to be skeuomorphic as developers start experimenting in this new design space where they can build Web3xAI native apps for the first time—our goal is to enable 1000 flowers to blossom. As it was hard for Vitalik Buterin to predict all of the use cases in 2014 with Ethereum, we are conscious that most use cases today have yet to be invented. Yet, here are some of the potential ones we foresee.

Agent swarms

It’s well-researched that multi-agent systems in complex tasks that require a set of fundamental abilities such as reasoning, multi-modality handling, web browsing, and general tool use outperform frontier LLMs on their own. On GAIA: A Benchmark for General AI Assistants by Meta, Hugging Face, etc., a multi-agent system performs almost 5x better than GPT4-Turbo, as seen on the GAIA Leaderboard.

Thus, Agent swarms can accomplish far more complex tasks when networked together. Swarm intelligence can also be observed in nature, such as in the case of bees, ants, and even humans. Large groups of relatively simple organisms can compute the solutions to complex problems by collectively organizing their behavior in such a way that the actions of one individual do not make much sense, but when put in the context of thousands of other individuals all following the same rules, complex computation takes place. Swarm intelligence follows a given set of steps in social animals, which shows the mechanisms at play in this type of meta-cognition:

  1. Information sharing: the information that each member obtains in its search needs to be shared with the whole swarm so that a decision can be made.

  2. Evaluation: a method for evaluating the information that has been shared must exit so that the swarm can make meaningful decisions. In the case of the honeybee, this method is built in how bees choose which dancing bees to follow. If there are several dancing bees, which one should each of the other bees follow into the new home? The answer is simple: the bee whose dance is more enthusiastic or convincing. Different bees seem to have various ‘opinions’ as to which of their mates makes a better ‘speech,’ giving rise to a sort of ‘elections’ in which there is no absolute consensus.

  3. Consensus building: In the case of bees, when a consensus has not yet been reached, and the difference in the ‘voting outcome’ is not that big (55% versus 45%), a ‘debate’ takes place. The bees start making loud noises and bumping against bees who have a different view from their own. In this manner, some of the bees that receive this type of treatment, as it is uncomfortable for them, change their mind about who to follow in favor of the view of the bee that is ‘confronting’ it. By this method, the swarm avoids deadlock between competing alternatives and ensures that the option that is to be chosen by the majority of the bees is indeed the right one.

  4. Committing on the vote: Now that the colony has made a clear decision by majority vote, the whole swarm follows the winning bee to their new hive.

Today, multi-agent systems follow similar mechanics, and Web3 provides them native rails to enable swarm intelligence - like the DeFi Lego blocks moment where simple core primitives have enabled developers to build a complex interlocking financial system. In the long-term, there’s also a fundamental question of whether AGI will be reached through a singleton, i.e., a single model, or through sorts of swarm intelligence that can be observed through evolution. The existing research seems to point to the latter. We believe Galadriel can significantly contribute to finding that out as we plan to scale on-chain swarm intelligence to millions of interlocking and interacting agents. This, in turn, changes the L1 from a deterministic state machine to more of a living organism driven by adaption - the hallmark of intelligence.

Culture/Social

We loved when NFTs hit the mainstream because they brought culture on-chain and showed that Web3 can be more than just serious financial applications. AI opens up a new experimentation canvas for artists and developers alike. For example, imagine generative NFTs that are constantly changing and evolving. Or, imagine your favorite meme coin becoming alive and having a much deeper connection with it - might it be dogwifhat, Pepe, or Doge.

On-chain Midjourney: https://minter.galadriel.com/
We also imagine use cases where existing centralized products will be decentralized, which can lead to better products with the right incentives. For example, in the same way, Farcaster is building a user-owned social media platform, we envision developers building user-owned ChatGPT, Midjourney, Character.ai, etc.

On-chain ChatGPT:https://chatgpt.galadriel.com/

AI Gaming & Simulations

Imagine a world where AI agents live and interact with each other in a virtual environment, learning and evolving just as humans do. Imagine playing this game where the world is ever-evolving. Google created this kind of Sims-like game in a small scale and we envision developers to scale these sorts of simulations up to millions of agents inhibiting complex behaviours.

We also see games where AI agents compete against each other, and users can bet on winners, like hamsters.gg, or users can compete themselves with on-chain AI to win prizes. As an example, we built VitAIlik: an RPG game where users can battle with an on-chain AI version of Vitalik. Every game is new, as AI generates the lore and gameplay dynamically. Much more of future gaming will be dynamically generated by AI based on the users’ preferences, and we see many of them coming on-chain.

On-chain VitAIlik game:https://vitailik.galadriel.com/

Finance & Settlement Layer for AI

We believe, in the long term, AI will drive GDP growth in the world most through its ability to conduct business between companies on a global scale. Effectively, companies are going from operating at a human speed to machine speed. It’s like the Internet made information move at the speed of light, but now it’s for whole companies. For this, a native payment and settlement layer is necessary where AI agents can exchange money in a trustless manner for the exchanged services.

Secondly, imagine you are part of a DAO but the managing of the treasury is not trustless. We’ve seen the exact problem in many DAOs and this could for example be solved by an AI DAO.

An AI DAO where the treasury is managed by AI agents

Furthermore, we see endless opportunities where AI can improve today’s Web3 products or create new categories by giving AI money and thus increasing its utility:

  • Automated yield farming;

  • Agent-based prediction markets;

  • On-chain insurance run by AI;

  • Hedge funds managed by AI;

Let 1000 flowers blossom

As Vitalik Buterin couldn’t foresee many of today’s use cases, such as NFTs, we imagine most of the use cases are yet to be dreamed of and invented. That’s why our goal is to enable developers to play with this new design space and experiment easily so that 1000 flowers can blossom.

Roadmap

Move fast, ship to users

We as a team have gone with multiple AI & Web3 startups from 0 → 1 and afterwards helped to scale to unicorns. Thus, we believe in the ethos of moving fast and iterating against users vs trying to come up with a perfect solution without grounding in reality. Thus, we err on the side of shipping early which also means if you are developing on top of Galadriel during devnet expect things to evolve and change rapidly.

Research

Besides, the core product there’s fundamental research to be conducted on the following areas: enabling privacy for user data, scalability in verified AI inference, bringing embeddings on chain, AI safety research and more.

Devnet

Galadriel’s Devnet is out now. During devnet developers can build working on-chain AI agents and applications. The focus is on developing the core technology, quickly iterating the product along with user needs and progressively decentralizing. We expect to make many big architectural changes during the devnet as the full chain spec is yet to be implemented.

Testnet

After the validation of the design and architecture during devnet, we aim to lock the chain spec and release a testnet. Biggest upgrade will be in bringing AI inference on the validator level so the network can achieve highest security and verifiability guarantees. It will act as the final testing ground before mainnet together with a security audit. We seek to reward and incentivise the early adopters and community members.

Mainnet

Following the successful iterations and invaluable feedback garnered through our testnet phase, we will launch Galadriel’s mainnet. In line with our tradition of rewarding early adopters and community members, the mainnet launch will distribute incentives and introduce new reward mechanisms.

Vision of Ethereum for AI

Galadriel’s mission is to build safe, citizen owned AI by building ETH for AI. Our long-term vision can be unpacked as:

  1. Build the first L1 for AI which acts as the settlement layer for AI, driving real GDP growth.

  2. Provide direct ownership and democratic governance over AI.

  3. Become the most secure rails for AI deployment and largest applied safety network where crypto-incentives drive humanity to align AI.

What is the future of AI? Will it be closed, corporate-owned, or safe and user-owned? Become a Galadrian and join the mission in making the latter happen.