SiloRepo (www.silorepo.com)
Generate Your Engineering Portfolio with LLM's
I co-founded Silo to take GitHub repos, PDFs, and slides and turn them into structured, multi-level engineering portfolios. Under the hood, it uses LLM pipelines + vector stores (OpenAI embeddings + Pinecone) to handle massive uploads, recursively chunking repos into navigable summaries. On top of that, I added a social network for engineers, making portfolios something you could share and browse like LinkedIn but focused on what you actually built.
Implementation
The core of the system was a multi-step LLM pipeline. We designed agents that operated in stages: chunking artifacts, embedding them into Pinecone, retrieving relevant slices, and then generating summaries at different abstraction levels (file → repo → portfolio). Each stage had a narrow role, and outputs from one became structured inputs to the next. This avoided token overflows, cut costs, and made the process modular. We also built eval hooks at each stage to check coverage and consistency, so summaries aligned across levels.
Technical Details
On top of that, we experimented with stylistic specialization. Some agents produced terse implementation details, others generated higher-level narratives, and others focused on “why this matters.” Portfolios combined these views into layered outputs. The backend was (largely) model-agnostic, we integrated OpenAI, Llama, and Groq, so we could tune for latency, cost, or fidelity. This gave us flexibility to run large-scale batch processing while keeping the system stable and outputs coherent.
Where is it now?
On the business side, we scaled SiloRepo to thousands of active users at Columbia SEAS, where it became the default way students shared their projects. The system handled hundreds of simultaneous uploads per week and processed repositories with hundreds of files without breaking. Mixpanel analytics gave us detailed usage metrics, feature adoption, portfolio views, and share rates, which we used to iterate quickly.
High Performance Research (Spec Decoding Paper)
SpecAdapt: Adaptive Speculative Decoding for LLM Inference
SpecAdapt was research project on accelerating LLM inference with speculative decoding. Traditional autoregressive decoding is slow because each token requires a full forward pass. Speculative decoding improves throughput by letting a lightweight “draft” model propose multiple tokens while a stronger “verify” model checks them. I implemented and benchmarked several approaches (Medusa, EAGLE, fuzzy SD, draft-and-verify) inside a PyTorch + vLLM harness, profiling GPU performance with PyTorch Profiler and Nsight to measure tokens/sec, SM utilization, memory transfers, and kernel overhead.
Prompt-Aware and Resource-Aware Routing
The second component was prompt-aware routing. We found that acceptance rates and efficiency vary widely by prompt type, code, natural language, or long contexts behave differently under speculative decoding. To handle this, I built a clustering-based routing system that embedded and classified prompts, then dynamically assigned decoding strategies (aggressive vs. conservative draft lengths). This increased throughput by ~1.6× compared to static speculative decoding, and we extended it with resource-aware routing that adapted draft length based on GPU load.The second component was prompt-aware routing. We found that acceptance rates and efficiency vary widely by prompt type, code, natural language, or long contexts behave differently under speculative decoding. To handle this, I built a clustering-based routing system that embedded and classified prompts, then dynamically assigned decoding strategies (aggressive vs. conservative draft lengths). This increased throughput by ~1.6× compared to static speculative decoding, and we extended it with resource-aware routing that adapted draft length based on GPU load.
Evaluation and System Insights
Beyond performance, I built a full evaluation harness to analyze coverage, latency distributions, and acceptance probabilities across workloads. We also experimented with KV cache compression and pruning to reduce wasted compute during rejected drafts. The key insight was that speculative decoding isn’t one-size-fits-all: adaptive strategies that account for prompt structure and hardware state can deliver real speedups, and SpecAdapt demonstrated that in practice.
Systems and Generalization
SpecAdapt wasn’t just about one method, it's related to how speculative decoding can be treated as a system-level optimization problem. By integrating multiple SD methods into a common harness, adding eval hooks, and making routing decisions adaptive, we demonstrated a framework that generalizes across models, workloads, and hardware constraints. The modular design means new decoding strategies or verification models can be swapped in without re-engineering the pipeline, making it a practical path forward for production inference systems.
Past Work Experience
