TechPrufer
Get started
GitHub
GitHub logoGitHub · Feature trail

Authentication Token Format Security

3 posts · Oct 2018 – Apr 2021

What this trail covers

This feature thread tracks the evolution of authentication token formats to enhance security and identifiability. Initially, tokens were hex-encoded 40-character strings with limitations for secret scanning. This post details the introduction of identifiable prefixes (ghp, gho, ghu, ghs, ghr) with an underscore separator, a checksum for offline verification, and increased token entropy. These changes aim to significantly reduce false positives in secret scanning and improve overall token security. The previous iteration focused on extending scanning to cloud provider tokens and unencrypted SSH keys, replacing assembly-based Git patches with a Go scanner leveraging the Hyperscan library for improved performance and extensibility.

All GitHub trails
  1. Behind GitHub's new authentication token formatsIntroduced new authentication token formats with identifiable 3-letter prefixes (e.g., 'ghp' for personal access tokens), an underscore separator for improved readability and double-click selection, and a 32-bit CRC32 checksum encoded in Base62 for offline verification. These changes increase token identifiability, reduce secret scanning false positives to an anticipated 0.5%, and increase token entropy (e.g., from 160 to 178 bits for OAuth tokens).
  2. GitHub Token Scanning—one billion tokens identified and five new partnersThis post announces the expansion of GitHub's token scanning partnership to include Atlassian, Dropbox, Discord, Proctorio, and Pulumi. It details the scale of the token scanning operation, having sent one billion tokens to partners for validation. The post also explains the technical workflow: scanning commits within seconds of push, notifying service providers upon detection, and provides an example of a Discord token notification. It also includes a call to action for other service providers to join the program.
  3. Behind the scenes of GitHub Token ScanningThis post details the evolution of GitHub's token scanning capabilities beyond just GitHub OAuth tokens. It describes the limitations of the previous assembly-based scanner and the transition to a more extensible, Go-based scanner utilizing the Hyperscan library. The post also outlines the private beta program with cloud service providers to test the scanning of various cloud credentials and unencrypted SSH private keys, and announces the public beta of this enhanced token scanning feature.