Home / Blog / Article

Gemini 4 Neon Benchmark for Agentic Coding

Gemini 4 Neon benchmark results for agentic coding and automation, with Antigravity comparisons, thinking levels, and the impact of prompt caching on costs.

🤖 AI & Automation Published on November 21, 2026 | Read time: approx. 12 minutes | Author: Pragma-Code Editorial
A digital assistant coordinates code review and testing at a software development workstation.

Google positions Gemini 4 Neon between Hydrogen and Carbon. Engineering teams also need to account for the agent harness, the cost of reusing context, and the quality of changes that pass review.

Part of our Themen-Hub series:

This article is an in-depth expert contribution from our content cluster. Discover the complete overview on our main page: AI & Automation →

Gemini 4 Neon benchmark at a glance

Google introduced Gemini 4 Neon on November 18, 2026 for demanding work that teams do frequently. Within the Gemini 4 family, Hydrogen is designed for fast execution, Neon for complex everyday tasks, and Carbon for the most challenging long-running work. Our article on Gemini 4 Argon and Frontier Intelligence covers the upper end of the model family. The question for Neon is which configuration makes recurring repository work economical once a team accounts for the work required to review and accept each change. Google · Model announcement

Key findings

  • General model performance. Neon on xhigh scores 61 points in version 5.0.1 of the Artificial Analysis Intelligence Index, placing third in the release comparison. Artificial Analysis · Release analysis
  • Coding-agent performance. Google reported a score of 58.6 and first place in the Coding Agent Index at launch. The configuration pairs Neon on xhigh with Antigravity CLI. Google · Launch, Artificial Analysis · Agent comparison
  • Standard API pricing. Google charges $3.00 per million standard input tokens, $0.15 per million cached input tokens, and $15.00 per million output tokens including thinking. Google · Developer API pricing

November 2026 release snapshot

gemini-4-neon shares a context window of roughly one million tokens across input and output. The model reference lists a default synchronous limit of 65,536 output tokens, subject to the remaining context. The lower input rate requires an implicit cache hit. Antigravity introduced the Neon–Hydrogen pairing with CLI v1.5.11 and added API-key selection for Neon in v1.5.12 on November 19. The Gemini Developer API uses the paid tier. Google lists Pro and Ultra as subscription access routes for Antigravity and the Gemini app at launch. Gemini API · Model reference, Google · Pricing, Antigravity · Changelog, Google · Availability

Coding benchmarks and what they measure

A benchmark offers a starting point for choosing what to test. Inside a project, the useful question is whether the configuration completes the task correctly and whether its patch holds up in review. Teams maintaining established interfaces will benefit from tasks that exercise dependencies across modules. Teams spending much of their time keeping builds and tests working may find terminal workflows more representative of their day-to-day effort. Match the benchmark to the work before using any headline result to narrow the field.

Artificial Analysis Coding Agent Index

Artificial Analysis gives equal weight to the components of Coding Agent Index v2.1. FrontierCode-AA 2.0 covers implementation tasks, Terminal-Bench 4.1 covers terminal workflows, and SWE-Atlas-QnA-AA covers questions about real repositories. Those tasks exercise different skills even when they use the same repository. Tracing an unfamiliar call path and explaining its purpose requires a different sequence of actions from writing a change and testing its behavior. Artificial Analysis · Coding Agent Index methodology

The configuration includes the model, the thinking level, and the agent harness. The harness manages tool calls and state, including how terminal output is passed back to the model. The rounded scores of 59 for Neon and 58 each for Fable and Opus apply to the configurations in the widget. Evaluate time and API cost per task on the same sample if those measures will influence your selection. Artificial Analysis · Release comparison

Benchmark comparison by metric

Rounded scores in Coding Agent Index v2.1
ModelPointsThinking level and harnessSource
Gemini 4 Neon59xhigh · Antigravity CLIArtificial Analysis
Claude Fable 5.258xhigh · Claude CodeArtificial Analysis
Claude Opus 5.558max · Claude CodeArtificial Analysis

Google reports 58.6 and first place for Neon at launch on November 18, 2026. The bars use the rounded Artificial Analysis scores. Google · Launch, Artificial Analysis · Coding Agent Index

Terminal-Bench 4.1 · Cheating-adjusted Accuracy
ModelScoreHarnessSource
Gemini 4 Neon65.2%Antigravity CLI v1.5.9DeepMind
Claude Opus 5.565.8%Claude CodeDeepMind
GPT-6.2 Sol61.5%CodexDeepMind
Additional benchmarks with their own scoring scales
BenchmarkConfigurationResultSource
FrontierCode 2.0Neon (high), Opus 5.5, GPT-6.2 Sol46.2% / 47.3% / 43.8%Cognition, DeepMind
SWE-sweepNeon · Antigravity CLI v1.5.939.0%DeepMind
SWE-sweepNeon · mini-SWE12.8%DeepMind
AutomationBench 1.0.6Neon · xhigh56.3%Zapier
AutomationBench 1.0.6Neon · high54.3%Zapier

The bars show Terminal-Bench 4.1. The table below the chart data gives the FrontierCode main score, SWE-sweep subtask-weighted reward, and Zapier end-to-end completion as separate metrics.

Gemini Developer API · Standard pricing per million tokens
Billing categoryPriceConditionSource
Input$3.00Standard inputGoogle · Pricing
Cached input$0.15Implicit cache hitGoogle · Pricing
Output$15.00Includes thinkingGoogle · Pricing

The cache rate requires an implicit hit. The output rate covers visible output and thinking. Google · Standard pricing

Terminal-Bench, FrontierCode, and SWE-sweep

Terminal-Bench 4.1 tests whether an agent can carry a terminal task through to a verified outcome. DeepMind reports cheating-adjusted accuracy. Neon scores 65.2% with Antigravity CLI v1.5.9, while Opus 5.5 scores 65.8% and GPT-6.2 Sol scores 61.5% in the reported configurations. For your own evaluation, record where unsuccessful runs break down. A missing build step and a misread test result may both cause a failure, yet they tell the team different things about the underlying workflow. Terminal-Bench · Version 4.1, DeepMind · Evaluation methodology, DeepMind · Model card

FrontierCode 2.0 focuses on implementing repository changes. The main score is 46.2% for Neon on high, 47.3% for Opus 5.5, and 43.8% for GPT-6.2 Sol. Artificial Analysis reports its own implementation and index component separately. SWE-sweep shows why the runtime matters. DeepMind reports 39.0% for Neon with Antigravity CLI v1.5.9 and 12.8% with mini-SWE, using subtask-weighted reward. Those results describe complete configurations. To measure the effect of one harness feature, change that feature and hold the rest of the setup constant. Cognition · FrontierCode 2.0, DeepMind · Model card, DeepMind · Evaluation methodology

Gemini vs Claude Sonnet vs GPT-6

Artificial Analysis reported 56 Intelligence Index points for Claude Sonnet 5.5 at max in its September 28 release evaluation. The Sonnet terminal result in that report uses Terminal-Bench 4.0 and belongs in a separate comparison from the Opus and Sol results above. Claude Fable 5.2 and GPT-6.2 Astra, both at max, rank ahead of Neon in the November release comparison of the general Intelligence Index. Our Fable and Astra benchmark comparison covers those two options in more detail. Artificial Analysis displays a rounded score of 60 for both Neon and Carbon on high. The Neon model page gives 44 as the median for its comparison group. Artificial Analysis · Sonnet evaluation, Artificial Analysis · Neon release comparison, Artificial Analysis · Google models, Artificial Analysis · Neon model page

For a comparison of multi-step tool use, start every candidate from the same repository commit. Ask each one to investigate a failed build, propose a narrow correction, and submit it for independent test verification. This reveals whether an agent falls short at understanding the repository, using the available tools, or producing a patch that satisfies the acceptance criteria.

Automation, science, and multimodal analysis

Zapier AutomationBench 1.0.6 tests whether agents finish simulated business tasks that span multiple applications. Zapier reports 56.3% for Neon on xhigh and first place in its release report. The default high setting scores 54.3%. This measures whether the actions produce the required outcome in the simulation. Production workflows still need clear ownership and dependable data. In Andon Labs’ Vending-Bench 2, Neon leads the leaderboard at the time of review with a simulated closing balance of $16,843.41 and reported variation of ±$972. The simulation tracks decisions over time about purchasing, inventory, and pricing. Zapier · Release report, Andon Labs · Vending-Bench 2

DeepMind reports a 70.5% average reward for scientific terminal tasks in Terminal-Bench Science 0.1, a score of 84.0% on mathematical reasoning in RiemannBench, and 71.3% on the demanding knowledge and reasoning questions in HLE-Diamond. In Chartography, Neon scores 73.5% on chart interpretation without external tools. In Agents’ Last Exam v1.1, DeepMind reports a 75.8% average rubric score for knowledge work in its Antigravity CLI v1.4.8 run. Each result reflects its own workload and scoring. Reading a chart correctly, for example, gives limited evidence about how an agent will handle an ambiguous table and its footnotes. DeepMind · Evaluation results, DeepMind · Test configurations

Prompt caching and LLM costs

Inference costs and cache hits

Prompt caching allows the provider to reuse work on previously processed input. Stable repository instructions and reference documents can form a shared prefix, followed by the details specific to the task. Use request-level usage data to measure actual reuse. Reordering files or changing instructions can shorten the reusable prefix. A cache miss is charged at the standard input rate. Neon supports implicit caching, with a 95% lower price for each input token billed as cached. Billable output includes thinking tokens that the user does not see. Pricing tab in the widget, Google · Standard pricing

Worked example for a repository workflow

Assume a hypothetical workflow across multiple API calls uses a total of 0.6 million uncached input tokens, 0.4 million cached input tokens, and 0.1 million billable output tokens. With absolute token counts the formula is (uncachedInput × 3 + cachedInput × 0.15 + billableOutput × 15) / 1_000_000. That comes to $1.80 for standard input, $0.06 for cached input, and $1.50 for output, for a total of $3.36. Billing the same input entirely at the standard rate brings the total to $4.50 for the same output. The volumes are assumptions for the example. Google · Pricing

To find cost per accepted task, add the cost of every attempt, including retries and abandoned runs, and divide by the number of results that meet the acceptance criteria. Include external tooling and CI as separate line items. Record human review time in minutes and apply an agreed internal rate if you need a fully loaded cost. That gives the team a way to see when a run saves API spend but increases the work required to approve a change.

Four thinking levels

low may be enough for a narrowly scoped classification. medium is a candidate for a small edit with unambiguous tests. high is the default and a sensible starting point for tasks with dependencies between steps. xhigh is worth testing when a task requires reconciling conflicting evidence across modules. Google Antigravity reports 32% lower cost per task for Neon on high compared with Carbon in its launch comparison. The announcement gives limited detail on its costing setup. The Artificial Analysis release article makes a separate comparison using Carbon’s undiscounted standard rates. Gemini API · What’s new in Neon, Antigravity · Release comparison, Artificial Analysis · Cost per task

When to use high or xhigh

Start on high using the task and harness the team expects to use in practice. If a multi-step diagnosis lacks evidence that a reviewer can verify, allow a bounded comparison on xhigh. Agree on the quality target, time limit, and spending cap in advance. End the run when it reaches a limit or further steps stop producing verifiable evidence. Compare additional quality and effort on the same tasks.

Agentic coding in 2026 with Google Antigravity

Neon plans, Hydrogen executes

Agentic coding combines planning, tool calls, and checks on intermediate results. Google describes Orchestrator as a single session in which Neon plans and Hydrogen handles routine edits and tests. If Hydrogen encounters a blocker, it returns the working state to Neon to reassess the plan. Google announced /swarm separately as a way to coordinate agents working in parallel. The diagram uses those roles in an illustrative internal repository maintenance workflow. The team decides which work can proceed in parallel based on dependencies between tasks. Google Antigravity · Orchestrator, Google Antigravity · /swarm

A ticket to maintain an internal library defines the scope. Neon plans the work, recording what tests should pass and which changes the team has authorized. Hydrogen tasks return test output, a documentation draft, and a narrowly scoped patch. Neon summarizes deviations. A reviewer examines the diff and approves integration. The team owns that approval process. For more on coordinating the work, see Multi-Agent Systems in Software Development.

Neon plans the work, Hydrogen executes subtasks, and a human reviews changes before integration.
Illustrative repository workflow. Hydrogen returns tests, documentation, and a patch to Neon. A human approves integration.

Task ownership

Orchestration and swarm

Give each worker a task with clearly assigned files and an expected deliverable. Documentation and test analysis can proceed concurrently if they use the same approved repository state. Sequence changes that touch the same location. When a worker reports a blocker, give Neon the working state along with the missing prerequisite.

Source verification

Multimodal analysis

A ticket may contain logs, screenshots, and earlier design documents. Ask the agent to return each finding with a file name, a reference to the relevant passage, and the version it used. Spot-check straightforward findings during review. If a screenshot and a log disagree, revisit the evidence and record the uncertainty in the ticket.

Secure changes

Cyber defense and hardening

Google reports 51.26% on WebExploitBench v1.2 and 76.0% on the AutoBaxBench Hard split, describing both as state of the art for their respective benchmarks at launch. DeepMind’s results table also lists 90.5% on SEC-bench Pro and 70.7% on CWE-bench v2. In the pilot, check generated code for secure implementation and put proposed changes through human patch review and regression tests. Google · Cyber defense launch results, DeepMind · Model card, DeepMind · Methodology

Budget and approval

Thinking-level steerability

Record the thinking level and its time and spending limits in the work order. When dependencies make the impact of a change unclear, the reviewer decides whether the available evidence justifies increasing effort. Give the new run the same acceptance criteria and record its cost separately.

From pilot to production

Begin with work the team already encounters. Put the same tickets through the existing process so reviewers can identify which work the agent takes on and which work the team still has to do. The pilot should end with a decision about each class of task the team might approve for production.

  1. Define scope and approvals

    The engineering lead and data owners select internal repositories and approve the tasks, data, and tools. Record which paths the agent can read or edit, what test environment it can use, and who approves exceptions. Agree on which findings can be retained in work logs and how long the team needs those records. For each ticket, have someone describe the expected outcome before the run starts. Give the pilot team a versioned task list and permission matrix that also apply to runs that are aborted or restarted.

  2. Measure the baseline and alternatives

    QA uses the same tasks for the current workflow, Neon on high, and the alternatives. Pin the starting commit, tools, test data, and acceptance criteria. For each attempt, record correctness, time to approval, human review minutes, token categories, and costs including retries. Break down the results by task type. An average across different work can hide a configuration that performs well on routine edits but leaves reviewers with substantial rework on changes that reach across modules.

  3. Run a controlled pilot

    The engineering lead runs approved tasks on isolated branches and checks diffs, tests, and tool logs. Stop when a run exceeds its budget, attempts an unauthorized write, or still lacks evidence after the permitted retry. Link each deviation to a ticket and record who reviewed it. Check which tasks produced verifiable results and which required human rework. The engineering lead uses the agreed quality and cost criteria to decide whether each type of work can move forward.

  4. Approve and monitor production use

    The service owner documents the approved scope, review requirements, spending limits, and how to return to the existing workflow. Start with the groups of tasks whose results were reproducible in the pilot. Track failures, aborted runs, and review time by task type. Repeat the relevant evaluation sample after changes to the model, thinking level, harness, or permissions. For a rollback, assign responsibility for stopping agent access, taking over active work, and recording the handover in the ticket.

If QA and engineering regularly disagree about whether tickets pass, clarify the acceptance rule before collecting another round of measurements. This matters when an explanation sounds plausible or a large diff makes it difficult to see whether the expected behavior changed. For a short pilot, choose tasks with documented tests and established review requirements.

Conclusion and pilot decision

Neon belongs on the shortlist for demanding repository work if the team can evaluate Antigravity or a comparable harness and subject changes to a review with traceable evidence. Public results can help identify tasks for the first round of testing. The team’s economic comparison depends on its own task mix, the cost of each attempt, and the work needed for approval. An inexpensive API run can trigger another round of review if it omits a regression test. A larger thinking budget can be worthwhile on a difficult investigation if it produces the evidence the reviewer needs.

Give the pilot a question with a measurable answer. For example, can Neon produce as many correct patches for isolated library changes as the existing workflow while keeping human review within the allotted time? Agree in advance how to treat missing tests, late results, and human rework. That gives the team something it can act on even if the candidates are tightly grouped in public evaluations.

Evaluation checklist

Record model, thinking level, and harness for every run.
Use the same representative task sample for each configuration.
Track cache hits, thinking costs, and retries.
Enforce patch review and defined tool permissions.
Document approval, ownership, and how to roll back.

Make the next decision by task type. Engineering and QA approve the groups of tasks that meet their quality, time, and cost criteria. Where high meets those criteria, use it as the baseline for subsequent changes to the setup. A bounded trial on xhigh is most informative when the team can state what evidence is missing and check it independently once the result comes back.

Official Sources and Primary Documentation

AI automation for your business?

I deliver this for small and mid-sized businesses: at a fixed price agreed upfront, working directly with the person who builds it. Process, scope and pricing are on the service page.

See service & pricing Or request a free intro call

Extended Specialized Glossary

What is Agentic Coding?

An AI agent plans and carries out software tasks in multiple steps. It uses tools to access repositories, make changes, and run tests.

What is an Agent Harness?

The runtime determines how a model uses tools, manages state, and executes tasks. Its configuration affects benchmark results.

What is Prompt Caching?

Prompt caching reuses previously processed input context. A cache hit may qualify those input tokens for a lower price.

Alexander Ohl

Alexander Ohl

Automated assistant • Online

Hello! I am the Pragma-Code Assistant. How can I help you today? You can ask me about our services or select a topic below.