#349

GitHub: `git clone`, `git fetch`, and `git push` over smart HTTP (feature request)

Openwaynesun09opened 6d ago

Problem

The GitHub emulator serves the REST API but not the Git smart HTTP protocol, so git clone, git fetch, and git push fail against an emulated repository. Tests that check out a repository, or push a branch and open a pull request through the API, still require real GitHub. Verified on main at b75beb7 with npx emulate@0.13.0.

Reproduction

Seed octocat/hello-world with auto_init: true and a repro_token for octocat, then start the GitHub service:

npx emulate@0.13.0 start --service github --port 4001 --seed emulate.config.yaml

Clone with the token as the password, matching how CI clients authenticate to GitHub:

git -c credential.helper= clone http://x-access-token:repro_token@localhost:4001/octocat/hello-world.git clone-try

Result (Git 2.50.1):

Cloning into 'clone-try'...
fatal: repository 'http://localhost:4001/octocat/hello-world.git/' not found

Both Git discovery requests return 404:

curl -s -i 'http://localhost:4001/octocat/hello-world.git/info/refs?service=git-upload-pack' -u x-access-token:repro_token
curl -s -i 'http://localhost:4001/octocat/hello-world.git/info/refs?service=git-receive-pack' -u x-access-token:repro_token
HTTP/1.1 404 Not Found
{"message":"Not Found","documentation_url":"https://emulate.dev/github"}

GitHub behavior

GitHub serves every repository at https://github.com/<owner>/<repo>.git (the .git suffix is optional) over the smart HTTP protocol:

  • GET /<owner>/<repo>.git/info/refs?service=git-upload-pack and POST /<owner>/<repo>.git/git-upload-pack handle clone and fetch operations. GitHub supports protocol v2 (Git-Protocol: version=2) and protocol v0.
  • GET /<owner>/<repo>.git/info/refs?service=git-receive-pack and POST /<owner>/<repo>.git/git-receive-pack handle push operations.
  • Authentication uses HTTP Basic with a token as the password. For GitHub App installation tokens, the documented username is x-access-token. Private repositories and all pushes require a token with access. Without one, GitHub returns 401 with WWW-Authenticate: Basic realm="GitHub", which prompts Git for credentials.
  • A push updates refs, enforces branch protection rules, sends a push webhook, and makes new commits visible through the REST API (commits, contents, compare, and pull requests).

Docs: https://git-scm.com/docs/http-protocol and https://git-scm.com/docs/gitprotocol-v2

Proposed scope

  • Clone and fetch of branches and tags over protocol v2, with protocol v0 as a fallback.
  • Push of new branches, fast-forward updates, and branch deletions through git-receive-pack. Reject a non-fast-forward update unless forced, and apply the same branch protection rules and permissions as REST API writes.
  • HTTP Basic token authentication that reuses existing token and installation token checks, returning the 401 challenge above.
  • A push webhook for each updated ref, matching the payload Contents API writes already send.
  • Out of scope for a first version: shallow clones (--depth), partial clone filters, Git LFS, and SSH.

Proposed implementation

The emulator already stores repository data as real Git objects. gitObjectSha hashes blobs, trees, and commits with the standard Git object header, and treeContent and gitCommitContent produce canonical object bytes, so object IDs match what Git computes:

https://github.com/vercel-labs/emulate/blob/b75beb7ce7d78676d471fff47432b77be6ff6528/packages/@emulators/github/src/git-helpers.ts#L6-L109

A smart HTTP route therefore needs to:

  1. Advertise refs from the existing ref and branch store.
  2. Walk the commits, trees, and blobs a client is missing and write them as a packfile of zlib-deflated whole objects, with no deltas in a first version.
  3. On push, parse the received packfile into blobs, trees, and commits, store them with the same helpers, then update refs under the same checks as PATCH /repos/:owner/:repo/git/refs/:ref.

This keeps clone and push consistent with the REST view of the same repository and requires no git binary on the host.

Activity

0

No activity yet.