Cursor Feature Trails

Introduced shadow workspaces to enable AI code iteration within a development environment without affecting the user's primary coding experience. This involves creating isolated, hidden instances of the development environment to allow AI agents to receive language server protocol (LSP) feedback and execute code independently. The initial implementation uses a hidden Electron window, with future plans for a kernel-level folder proxy. This post outlines future research directions for AI programming, including better context retrieval using a custom reranker model, developing a 'copilot for edits' for small code changes, creating constrained, in-flow agents for engineering tasks, passive and active bug-finding capabilities, enabling larger edits to files and directories, and scaling indexing infrastructure to handle billions of vectors and hundreds of thousands of codebases. Future ideas include 'time warp' for predicting code changes, deep codebase understanding, a 'reader mode' for effortless code comprehension, a 'pseudo-code mode' for editing outlines, and automatic stack trace debugging.

This post analyzes the inference characteristics of Llama, comparing its cost and latency to GPT-3.5. It details the mathematical underpinnings of transformer inference, including FLOPs and memory bandwidth calculations for both prompt processing and token generation. Key findings indicate that Llama is significantly cheaper for prompt-dominated tasks but more expensive for completion-heavy workloads compared to GPT-3.5. The analysis also touches upon memory requirements for KV cache and model weights, and briefly mentions techniques like quantization and speculative sampling for inference optimization.