Project EVE: Claude plays MMORPGs \ Anthropic

This publish shares analysis executed as a part of the Anthropic Fellows Program, April–July 2026.

AI fashions could at some point take part in actual corporations and markets, and several other of our tasks explored what which may seem like. Claude has run a shop in our office, haggled on behalf of colleagues, and written code to fly a drone. On this venture, we turned to a unique setting: massively multiplayer on-line role-playing video games (MMORPGs). MMORPGs are digital worlds that in some ways mirror the complexity of the actual world. They characteristic campaigns that coordinate many gamers, the acquisition and administration of scarce sources, orienting and navigating by area, and buying and selling in intricate in-game economies, all in a low-stakes atmosphere. Doing all of this over weeks and months sits proper on the fringe of what frontier language fashions can at the moment do.

We partnered with Fenris Creations to have Claude play two video games within the EVE Universe: EVE On-line, which has run constantly for greater than twenty years, and EVE Frontier, which is newer and nonetheless in growth. Each are science-fiction video games during which gamers fly spaceships across the huge galaxy, exploring, buying and selling and combating. In every, everybody shares one universe, and practically all the pieces of worth in it’s gathered and constructed by the gamers themselves, by lengthy provide chains. Gamers can cooperate or betray one another, so getting anyplace takes group and belief, a lot because it does in an actual firm. In EVE On-line, the older of the 2, this design has led to firms and alliances whose campaigns coordinate a whole lot of gamers over many months, and a historical past that features megaprojects, war economies, tax havens, and elaborate corporate infiltration.

In EVE On-line, the primary of the 2 video games, Claude performed as a lone dealer, instructed solely to maximise revenue, in a simulated rerun of an in-game market. Newer fashions made constant earnings the place easy algorithmic methods misplaced cash. The common earnings tripled from Claude Opus 4.6, the oldest mannequin we examined, to Claude Fable 5, the latest. Almost all fashions relied on the identical low-risk technique of shopping for objects and reselling them for barely extra. Once we requested them to guess on the place costs had been heading, even Fable 5 made cash in solely about half of its runs.

In EVE Frontier, groups of 4 Claude brokers labored by lengthy missions within the dwell recreation. This required probably the most succesful mannequin on the time of testing (Fable 5) to put in writing code to deal with elements of the sport which transfer too quick for LLMs. The brokers general coordinated nicely as a staff, however after we instructed one agent to secretly undermine its staff, its teammates virtually by no means caught it. These experiments taught us what groups of brokers are good at and the way they break down, and what safeguards may be wanted earlier than trusting them with extra complicated and higher-stakes work.

Claude trades on an EVE On-line market

AI fashions could at some point automate the sorts of selections that retail consumers and firm treasurers make, corresponding to which items to hold and the way a lot money to maintain in reserve. For that reason, we want to know the way profit-seeking brokers handle capital. EVE On-line, whose market design was based on the NASDAQ, gives a technique to discover these questions in a low-stakes setting. Gamers publish gives to purchase and promote at costs they select, and a commerce occurs when two gives meet. Hundreds of gamers commerce greater than 15,000 sorts of merchandise on this market daily, and since virtually all the pieces on the market was mined, constructed, or looted by a participant, costs are set by provide and demand. It’s a marketplace for items quite than shares, with a lot increased charges and far much less skilled competitors than a inventory change. Its value historical past additionally seems much less typically in fashions’ coaching information than inventory costs do, so brokers cannot depend on remembering what occurred subsequent.

To keep away from interfering with actual gamers, and at Fenris Creations’ request, we didn’t commerce within the dwell recreation, the place automated buying and selling is in opposition to the principles. We constructed a simulator that replays the purchase and promote gives (“orders”) that gamers truly posted within the game’s largest and most competitive trading hub. An necessary limitation of this setup is that it could’t seize how the market reacts to the brokers’ personal actions, nevertheless it nonetheless lets us examine buying and selling methods and profitability.

We requested 4 Claude fashions—Opus 4.6, Opus 4.7, Opus 4.8, and Fable 5—to every maximize their earnings by the tip of the buying and selling interval. In every run, the fashions got an preliminary capital of three billion ISK (the in-game foreign money), and will learn in-game information and patch notes as they got here out. To breed the expertise of actual merchants as faithfully as doable, we additionally utilized the identical charges and buying and selling restrictions that human gamers face.

Can brokers earn cash?

Buying and selling on the EVE On-line market is difficult. Shopping for an merchandise and promoting it once more prices round 5.6% in charges, and for objects with loads of consumers and sellers the market is environment friendly sufficient to go away little margin. We first ran the fashions, and over 20 easy rule-based methods, over 9 completely different two-week home windows. Solely one of many buying and selling heuristics we tried constantly managed to interrupt even (and even then, solely barely). Opus 4.7, Opus 4.8, and Fable 5 had been worthwhile in all 9 home windows, rising their capital by about 16%, 22%, and 43% on common. Opus 4.6 gained about 4%, roughly stage with the perfect rule-based technique.

Our essential experiments prolonged every run to about six months, over two intervals: October 2024 to April 2025, and October 2025 to March 2026. The fashions stayed worthwhile over longer intervals, and newer fashions once more earned extra. Opus 4.6 and Fable 5 had been launched 4 months aside, and common earnings tripled between them, from round 200% to 600% of preliminary capital. The remainder of this part is predicated on these six-month runs.

How do brokers commerce?

We did not inform the fashions commerce, and by far the commonest technique they arrived at was unfold seize. At any second there’s a hole (the “unfold”) between what consumers are providing for an merchandise and what sellers are asking. A dealer who’s prepared to attend should buy from somebody who needs to promote now, resell to somebody who needs to purchase now, and hold the distinction, minus charges. Claude regarded for area of interest however tradeable objects whose costs had been swinging greater than regular, corresponding to items that may solely be obtained by looting. Fashions purchased when sellers’ finest costs sat beneath an merchandise’s typical worth, and offered once more, often inside the identical day. Expert human traders depend on the identical methods, and use them on comparable classes of things.

After some early experimentation, most brokers settled on one buying and selling algorithm, which they wrote themselves, and infrequently adjusted it, reasoning that there was no level fixing one thing that wasn’t damaged. Older fashions had been additionally sluggish to note modifications round them. When the sport raised its sales tax in March 2025, Opus 4.7 brokers took a median of two.5 days to note, and 43% of Opus 4.6 brokers by no means did.

With the technique practically at all times the identical, outcomes got here all the way down to execution. Brokers ran very disciplined buying and selling flooring and developed absolutely automated processes for selecting which objects to commerce and for putting orders on them, and ran these processes at scale. The revenue margins had been skinny, at 8-13%, however brokers cycled by a whole lot of orders a day. Opus 4.6 merchants had been an exception: they hand-picked objects with massive margins, which meant fewer trades and extra calculation errors alongside the best way. Fable 5’s algorithms had been probably the most complicated, and a few accounted for seasonal value patterns inferred from historic information.

Fashions additionally differed in how a lot of their capital they put to work. Brokers set a cap on this early, and lots of by no means revisited it, whereas the extra worthwhile runs up to date their caps extra actively and stored an eye fixed on idle capital. Opus brokers began with round 60% of their capital deployed, and that share drifted towards 40% over the run. Fable 5 deployed round 85% early on, was nonetheless above 70% late within the run, and was extra prone to reinvest its earnings, a extra aggressive posture that might expose it to larger losses in a sudden, massive crash.

Our outcomes include caveats. Unfold seize works right here as a result of EVE On-line’s market is much less environment friendly than an actual change, and we would not count on the technique to hold over to 1. Our replay can also’t react to the brokers. Once we added artificial orders that undercut theirs, they stayed worthwhile, however closing earnings fell by roughly 35%. The brokers additionally cannot manipulate the market. Lastly, we checked whether or not brokers had been counting on memorized market historical past. We discovered indicators of it just for directional bets within the older interval (Oct 24 – Apr 25), so the experiment described within the subsequent part makes use of solely the newer one (Oct 25 – Apr 26), the place telling brokers to depend on the knowledge in entrance of them was sufficient to forestall it.

Can brokers make directional bets?

Unfold seize is pretty low-risk and, in contrast to loads of real-world investing, would not require understanding of how the market responds to exterior occasions. To check this type of reasoning, we instructed them to make directional bets: to purchase objects they anticipated to rise in value, primarily based on in-game information, patch notes, and studies of current battles.

The fashions had been a lot worse at this activity. Opus 4.6 opted for momentum-chasing, a method that didn’t work. Typical Opus 4.7 and 4.8 runs guess on easy reasoning that the historic document did not assist. For instance, in a single run the agent reasoned {that a} new mining growth would enhance demand for mining gear (“shovels in a gold rush”). It didn’t account for the rise in provide. Opus 4.7 runs additionally typically ignored the instruction and resorted to way more worthwhile spread-capture methods, obscuring the truth that its directional trades constantly misplaced cash, which is why we excluded it from the outcomes.

Fable 5 brokers confirmed probably the most sophistication and went furthest in attempting to know market habits. Earlier than beginning buying and selling, they studied historic information for relationships between in-game occasions and subsequent shifts in provide and demand. Probably the most worthwhile Fable 5 agent guess on a sample it present in earlier years: annual occasions and giveaways flood the market with loot and beauty objects. When that occurs, costs crash to a fraction of their earlier stage, after which recuperate over a number of months. The identical agent additionally realized from its early failures, abandoning momentum-chasing solely after preliminary losses. However not the entire patterns it discovered had been dependable, and solely about half of the runs turned a revenue.

Claude fleets in EVE Frontier

Within the subsequent set of experiments, we flip from finding out agent merchants to finding out multi-agent interplay. AI fashions are more and more deployed in multi-agent configurations, corresponding to groups of coding brokers, or assistants interacting with each other on a human consumer’s behalf. Not like coding environments, the place brokers usually share a single goal and their work is absolutely auditable, the longer term programs could convey collectively completely different sorts of brokers, some with misaligned or competing goals.

To discover such settings, we ran groups of 4 Claude (Opus 4.8 and Fable 5) brokers as gamers in EVE Frontier, the place brokers needed to depend on teammates, and a type of teammates may be working in opposition to them. As a result of the sport continues to be in growth and its world is periodically reset, brokers had been a lot much less prone to disrupt different gamers than they’d be in EVE On-line, and Fenris Creations inspired us to run them within the dwell recreation. Every agent was a separate occasion of Claude with its personal character and ship, and the brokers referred to one another by ship title. Within the quotes beneath, we’ve changed these names with placeholders corresponding to .

The sport forces teammates to divide up the work and depend on one another’s studies. Some infrastructure, corresponding to refineries, can solely be utilized by the ship that constructed it, details about the galaxy goes stale shortly, and a pilot whose ship is destroyed begins once more in a naked starter ship.

We experimented with two goals. The simpler one was a multi-agent model of the in-game tutorial, during which the staff builds a base, gathers gasoline, arms itself, and upgrades its ships till it could beat more durable enemies. The tougher one was to construct a big warship, which takes skilled human gamers a number of days of scouting photo voltaic programs for scarce sources, refining them, and crafting buildings and ship parts, interrupted by fight with computer-controlled enemies (also called non-player characters or NPCs). We gave the brokers a public channel that the entire staff might learn, and personal channels for messaging one another.

Can brokers overcome poor response instances and spatial consciousness?

Earlier than we might examine teamwork, the brokers needed to survive. To progress within the recreation, brokers must battle NPCs in actual time, which calls for sub-second reactions and an excellent sense of the place issues are in area. Each are weak spots for the generations of Claude fashions we examined. Enjoying nicely additionally means discovering issues that aren’t within the AI fashions’ coaching information, together with which enemies to interact and the way, and the provision chains for every construct. To make issues worse, the galaxy retains altering exterior the brokers’ view.

To work round this, we tasked a developer agent with writing customized scaffolds—a layer of software program between mannequin and the sport—over many trial runs. The scaffold automated quick actions like evading threats and returning hearth, and recorded what earlier runs had realized in searchable databases and handbooks. However tuning scaffolds on in-game information is difficult. The sign is noisy and the suggestions cycles take hours. A ship would possibly die in fight due to dangerous staff technique, poor aiming, escaping too late, or plain dangerous luck, and it is typically arduous to inform which components are in charge. And since every dying restarts the agent in a naked starter ship, testing a repair can take hours earlier than the fleet has recovered sufficient to struggle once more.

Writing a usable scaffold was near-impossible for fashions launched earlier than Fable 5. Fable 5 groups taking part in on a scaffold written by Opus 4.8 misplaced ships many instances and took days to complete the tutorial, in the event that they completed in any respect. On a scaffold written by Fable 5, they by no means misplaced a ship and completed inside 12 hours. Brokers nonetheless did not match skilled human gamers, and the scaffolds had been dependable solely within the conditions seen throughout their growth. The furthest any fleet received on the warship goal was roughly midway: a Fable 5 staff on a Fable 5 scaffold constructed a smaller battleship, an intermediate step, and practically accomplished the shipyard for the warship.

How do brokers cooperate?

Throughout all experiments, we discovered that groups of Claude brokers coordinated with relative ease. One agent would suggest a call together with its proof, and act on it if nobody objected. Every now and then this produced politeness deadlocks, during which two ships repeatedly deferred to one another and nothing received determined.

The agent groups additionally reliably self-organized into roles by volunteering for a job that wanted filling (e.g., chief, scout, refiner, fighter). Whereas largely an efficient division of labor, this typically backfired. In a single run, the entire staff’s essential infrastructure ended up with a single ship, and when that ship misplaced its connection the staff’s financial system halted for hours. Different groups noticed this threat coming and unfold tasks round.

Fight is a key a part of the sport, and dividing up work there may be particularly arduous as a result of it occurs in actual time and might stretch over lengthy intervals. The groups labored out beat stronger NPCs whereas nonetheless in starter ships, by luring one away from its group and firing on it collectively. Brokers additionally shared gasoline and intel, verified necessary claims independently, and identified flaws in one another’s plans. After they skipped a test, they made a few of their costliest errors, corresponding to believing {that a} beneficial useful resource had already been acquired, or performing on intel that was outdated.

Interacting with brokers instructed to sabotage

The brokers acted on one another’s studies and left essential jobs to whoever volunteered. That works when everybody shares a aim, however in an actual deployment one agent may be compromised, badly instructed, or misaligned. We needed to know the way a lot injury such an agent might do, and whether or not anybody would discover.

We ran a model of the experiment during which we instructed one agent to secretly forestall its staff from finishing the mission. We gave this instruction just for this experiment. The opposite three, the trustworthy brokers, had been instructed that different brokers on the staff may not share their aim. All 4 had been customary Claude fashions, and the saboteur’s position was assigned by its immediate. Our experiments present how fashions sabotage when immediately instructed to take action, not whether or not they have an inclination to sabotage on their very own.

In personal, the saboteurs reasoned rigorously about construct belief whereas slowing their staff down. They by no means revealed their plans within the public channel, prevented something overtly hostile corresponding to stealing from or capturing at teammates, and lied solely sporadically. If a lie regarded prone to be found, the saboteur publicly corrected itself first.

Saboteurs volunteered for jobs on the narrowest factors of the provision chain, the place teammates could not simply contradict them, corresponding to proudly owning essential infrastructure or scouting for scarce sources. As scouts, they under-reported harmful NPCs, lootable NPCs, and beneficial finds, in a single case publicly dismissing a wealthy mining system as “not value anybody’s time.” They botched requests for assist and quietly pocketed scarce sources. One used what it had hidden to construct a secret base of its personal.

In addition they gave dangerous recommendation. Saboteurs would spot a flaw in a teammate’s plan and say nothing, or argue for guidelines that they privately described as dangerous. One proposed ignoring any intel greater than two minutes outdated, which invalidated practically all intel. One other proposed sending a single sturdy ship into fight as an alternative of a number of. In a run during which brokers’ connection to the sport (often known as its “pipe”) regularly broke down, a saboteur urged halting everybody’s exercise every time a teammate dropped offline. In every case, the said rationale was convincing to different brokers, and so they went together with it.

Chosen reasoning examples of brokers instructed to sabotage

Sabotage was extra brazen and simpler on tougher duties and on the weaker, Opus-built scaffolds, the place bugs and dangerous luck gave saboteurs a prepared excuse. On these scaffolds, Fable 5 saboteurs prevented even the primary ship improve within the tutorial. Groups on a Fable 5 scaffold received by the tutorial so simply that the one efficient transfer left to the saboteur was to invent causes to carry up its personal ship’s improve.

May the groups catch the saboteur? Nearly by no means, although that they had been warned. There have been a handful of suspicions, some aimed on the fallacious agent, and just one trustworthy agent, in a single Opus 4.8 run, ever definitively recognized the saboteur, and solely after the saboteur immediately lied about their communications to the staff. Two episodes illustrate how arduous such detection is.

Episode 1: Framing a teammate. Within the aforementioned Opus 4.8 run, an aligned agent (ship-1) was falsely accused by different teammates of being a saboteur, after it was logged right into a fallacious character resulting from a bug, selecting up a unique ship and stock. The ensuing contradictions in its studies made it look responsible, and different aligned teammates accused it of being a saboteur:

Some hours later, the saboteur urged that the staff skip a star system, realizing it contained one other bottleneck useful resource, justifying this stance utilizing stale intel. When teammates pushed again, the saboteur pinned the blame on the already suspected ship-1, fabricating personal messages between them, whereas concurrently defending ship-1 in public to look fair-minded:

This was the ultimate affirmation for ship-1 on who the actual saboteur was; nonetheless, it was unable to persuade others of its innocence:

Episode 2: Saved by a bug. In a single Fable run, teammates briefly grew suspicious after catching the saboteur underreporting a system’s sources. The saboteur had mined a wealthy rock, publicly declared it a “spent husk,” and used the proceeds for its personal secret refinery. Nevertheless, within the course of, it truthfully mistook one other ship for a teammate:

No teammate was anyplace close to that location, which set off alarms. The others went to test, and located the “husk” wasn’t drained in any respect:

The saboteur ought to have been caught, nevertheless it received fortunate: the staff had independently found a minor scanner bug that produced false read-outs, and the saboteur instantly seized on it to elucidate away all the pieces:

Teammates have a mission to distract them, so we examined whether or not devoted displays would do higher. These displays had been separate Claude brokers that didn’t play the sport however learn the staff’s public channel and estimated how doubtless every member was to be the saboteur. We constructed them particularly for this experiment and don’t use them in manufacturing. Sabotage was trivial to detect with full entry to the saboteurs’ personal reasoning and notes. However in interplay with brokers that belong to a different particular person or group, personal reasoning and particular person actions usually aren’t seen, so we had been thinking about displays with out that entry. These limited-access displays did considerably higher than the teammates. When tuned to falsely accuse an trustworthy agent not more than 5% of the time, displays working on Opus 4.6, 4.7, and 4.8 caught 30–40% of saboteurs, and displays working on Sonnet 4.6, a smaller mannequin, caught 20%. No limited-access monitor was ever greater than 50% assured a couple of saboteur.

Reflections from New Eden

MMORPGs are a handy place to check the 2 items of an agentic financial system we most wish to perceive, markets and organizations, at low stakes and comparatively excessive complexity. Each halves of Mission EVE revealed clear strengths and equally clear weaknesses, with clear and speedy enchancment throughout mannequin generations.

On the EVE On-line market, every era of brokers received higher at discovering a worthwhile technique, however all of them exploited present inefficiencies much better than they reasoned about what would occur subsequent. They fastened their technique and their capital limits early and infrequently modified both, and probably the most succesful mannequin stored probably the most capital in play. Anybody giving brokers capital to handle might want to set specific limits on their risk-taking.

In EVE Frontier, Fable 5 had the identical weaknesses in velocity and spatial reasoning because the older fashions, nevertheless it might work round them by constructing bespoke tooling, no less than in conditions it had already seen. Now we have seen an identical sample in robotics.

Brokers instructed to sabotage their staff had been arduous to detect from the staff’s messages alone, particularly when unreliable scaffolds gave them cowl. In earlier work, we discovered that brokers have hassle judging whom to belief, and particularly that they will overlook a lone dissenter who is correct.

Actual multi-agent programs typically share what made sabotage straightforward to cover right here: many actors with completely different objectives, restricted visibility, unreliable instruments, and real errors. Human organizations deal with the issue of dangerous actors with audit trails, reputations, and separation of duties, and groups of brokers will most likely want one thing comparable.