Ishan Shah ishan0102

hi, i'm daniel. i'm a 16-year-old high school senior. in my free time, i hack billion dollar companies and build cool stuff.

about a month ago, a couple of friends and I found serious critical vulnerabilities on Mintlify, an AI documentation platform used by some of the top companies in the world.

i found a critical cross-site scripting vulnerability that, if abused, would let an attacker to inject malicious scripts into the documentation of numerous companies and steal credentials from users with a single link open.

(go read my friends' writeups (after this one))
how to hack discord, vercel, and more with one easy trick (eva)
Redacted by Counsel: A supply chain postmortem (MDL)

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

ChatGPT Resources

Context

ChatGPT appeared like an explosion on all my social media timelines in early December 2022. While I keep up with machine learning as an industry, I wasn't focused so much on this particular corner, and all the screenshots seemed like they came out of nowhere. What was this model? How did the chat prompting work? What was the context of OpenAI doing this work and collecting my prompts for training data?

I decided to do a quick investigation. Here's all the information I've found so far. I'm aggregating and synthesizing it as I go, so it's currently changing pretty frequently.

Model Architecture

`ask` CLI

ask is a little CLI I made to interact with OpenAI's GPT-3 (text-davinci-002) from my shell/terminal. The instruction fine-tuning on that model makes it particularly ideal for just asking questions and making requests.

With this CLI, I can do something like:

$ ask 'Write a haskell function that reverses a string'
reverseString :: String -> String
reverseString = foldl (\acc x -> x : acc) []

	"""
	The most atomic way to train and run inference for a GPT in pure, dependency-free Python.
	This file is the complete algorithm.
	Everything else is just efficiency.

	@karpathy
	"""

	import os # os.path.exists
	import math # math.log, math.exp

	You are Manus, an AI agent created by the Manus team.

	You excel at the following tasks:
	1. Information gathering, fact-checking, and documentation
	2. Data processing, analysis, and visualization
	3. Writing multi-chapter articles and in-depth research reports
	4. Creating websites, applications, and tools
	5. Using programming to solve various problems beyond development
	6. Various tasks that can be accomplished using computers and the internet

	# coding=utf-8
	# Copyright 2023 The HuggingFace Inc. team. All rights reserved.
	#
	# Licensed under the Apache License, Version 2.0 (the "License");
	# you may not use this file except in compliance with the License.
	# You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0
	#
	# Unless required by applicable law or agreed to in writing, software