How per-token code pricing works

AI labs buy code by the token, and a token here is four bytes of a file. Pangea pays $100 per million tokens of source code and $40 per million tokens of git history for a non-exclusive license. A typical line of code is about ten tokens, so a million tokens is roughly 100,000 lines. The rules below are copied from pre.dev’s published counter, the script Pangea’s pipeline runs, and the worth calculator applies them to your repositories in your browser.

Tokens are bytes divided by four, not words or lines

The counter reads file sizes, not language. A 2,000-byte file is 500 tokens whether it is Python, CSS or a README. That makes the price easy to check by hand:

Repository	Source bytes	Source tokens	Source value
A small script	40 KB	10,000	$1
A side project	400 KB	100,000	$10
A product	4 MB	1,000,000	$100
A large codebase	20 MB	5,000,000	$500

Those are the source figures only. The history figure is usually several times larger.

Source is the latest version; history is every earlier one

Source is each code file as it stands in the latest commit, counted once. History is each earlier version of each counted file, on every branch and tag, each distinct version counted once. A version identical to today’s file counts in source only. Commit messages, authors and dates are not counted. Each file path is capped at 100 MB across all its versions, about 26 million tokens.

So a file that was rewritten twenty times carries about twenty versions of itself in history, priced at $40 per million tokens, on top of its single source count at $100 per million. That is why Pangea says “Git history carries most of the value”, and why 79% of the tokens on its network are history.

Docs, config and data count, up to ten times the code

Code files (.js, .ts, .py, .go, .rs, .java, .c, .sql, .css and about sixty other extensions) count in full. Text files (.md, .json, .yaml, .html, .xml, .toml and similar) count too, but only up to ten times the bytes of code beside them, in source and in history separately. A repository that is one script and a 50 MB JSON dump is priced as the script plus ten times its size, not as the dump.

Dependencies, builds and minified files count for nothing

The counter skips whole folders by name: node_modules, vendor, dist, build, target, .next, __pycache__, .git, Pods and the rest of the usual dependency and build output. It skips files named like bundles, such as main.min.js or chunk.8f3a9c2e.js, and any file with a single line of 100 KB or more, which is the test for minified code. Pangea’s page adds that generated files, vendored code, binaries, data dumps and third-party code stay out of the valuation. None of this needs cleaning up first: Pangea takes repositories as they are and leaves the excluded parts out of the count.

Three worked examples

History can only be counted exactly by pre.dev’s script, run on a full clone. githubworth.com estimates it from source size and commit count, with a published error band, and these are its worked examples at Pangea’s rates:

Source tokens	Commits	Estimated history tokens	Low	Middle	High
50,000	30	161,653	$7	$11	$30
1,000,000	500	8.1 million	$187	$424	$1,331
5,000,000	3,000	84.8 million	$1,416	$3,893	$13,395

Pangea’s own illustration, roughly $1,000 to $1,500 per repository at typical depth, sits inside the last two rows. Repositories with ten commits or fewer are the least predictable, so for those the range matters more than the middle.

Per token means per copy, and a copy can sell more than once

The license is non-exclusive. You keep the code, and the same repository can be licensed to several labs, each paying at the published rate. An exclusive license or a full acquisition is a separate written agreement with its own price.

The rate is a list price; the grade and demand set the deal

Pangea calls its rates indicative and can change them on 30 days’ notice, never retroactively. Before anything is licensed, an AI lab reviews a scrubbed copy of each repository against a fixed rubric, and Pangea says the final value depends on that analysis and on buyer demand. Licensing typically takes 2 to 16 weeks and is not guaranteed, and you’re paid within 15 days of a lab’s payment. The token count tells you the ceiling at today’s rates. How much is my GitHub repo worth has the three ways to get your own count.