1Welcome back
Left off: Fellowship essay · draft 2
When you sit down after a long gap.An always-on assistant for Windows
Remembers what you saw. Speaks only when asked.
Jimmy turns your screen's text and the speech near your mic into one searchable history, kept on your own PC. Say "Jimmy, …" and it answers out loud, beside the moments it found. The rest of the time, it stays quiet.
- Captures stay on your PC
- English and Hindi
- Windows 10 and 11, no admin needed
A re-creation of the overlay with invented data. The real one floats over whatever you're doing and never takes focus.
The trailer
Jimmy, in under two minutes
What it remembers, how it answers, what it does on screen, and what it never keeps. Turn the sound on.
The idea
Capturing everything is easy. Staying quiet is the product.
Jimmy is built for six good interruptions an evening, not a stream of notifications. A model never writes a card: code writes every line from words in your own data, with real times.
4cards an hour, at mostA hard cap in code, over a rolling hour.
10min between two cardsNever two in a row, however much changes.
30min of quietAfter you dismiss one. Dismiss a kind three times in one app and it stops showing there.
An illustrative day · capture runs all day, cards come rarely
curtain down
2Recall
Same Northwind Fellowship as Tue 15:02
What's on screen, linked to a concrete earlier moment.3Focus
Back to: the essay
Only because you said "focus on the essay".4Your day
Today: 10h 42m, mostly VS Code
Once an evening. Ask "how was my day?" for the chart.Ask
Say it the way you remember it
No keywords and no app to open. Jimmy searches by meaning inside the time you name, shows the moments it used, and answers out loud in about a second. After it answers, keep talking for ten seconds without the name.
what was that fellowship application I saw on Tuesday?
Searches Tuesday by keywords and meaning; evidence on the left, answer on the right.how long was I on YouTube today?
Instant, with no model: one line written by code, plus a chart.remind me at 5 to call Sam
A card and a spoken reminder at five.forget the last hour
Says exactly what would go, then waits for your "yes".
Prefer typing? Ctrl + Alt + Space opens the pill as a search box.
Hands
Its own cursor, never yours
Ask it to do something on screen and Jimmy makes a short plan, plays it back live, and presses and types through Windows' accessibility interface. Every action is checked before the next. A policy in code decides what it may do, whatever model proposed it:
- RefuseTyping into a password box, or words you didn't say.
- StopA CAPTCHA. Jimmy never solves one.
- AskAnything that can't be undone, or a site you didn't name.
- AllowAn undoable action on a control you named, shown first, with a moment to say "stop".
You said "search for CarryMinati"; nothing in the plan was risky, so it started on its own.
In motion
Nine things, a few seconds each
Clips from the trailer. Pick one, or let them play through.
Private by construction
Absent code paths, not settings left off
Walk away and the webcam notices: the curtain comes down and Jimmy stops capturing and listening. Faces are blurred before a screenshot is written, and a test re-runs the face detector on the saved file to prove it finds none. What leaves your PC, and what's not done yet
Looking down at your phone doesn't count; leaving the camera's view for a moment and a half does. Ctrl + Alt + L draws it by hand.
Kept on your PC
- Window text and titles
- Blurred screenshots
- Transcripts of speech
- How many faces, never whose
Never kept
- An unblurred frame
- Raw audio, or the other side of a call
- Banking, password managers, private windows
- A faces table or a people table
Measured, not promised
Numbers from the laptop it was built on
Each one is written down in the repository with how it was taken.
Right first decision on 130 real spoken commands
124/ 130
Search over your whole history, keywords and meaning
0.5s
Faces the detector finds in a saved screenshot
0
The test puts 2 faces in, saves the blurred JPEG, and runs the face detector on it again.
One laptop: Ryzen 7 7840HS, RTX 4050 with 6 GB. Your results will vary.All measurements, with weak spots
The real thing
What it looks like on a screen
Screenshots of the actual overlay, showing invented pages and data, so no personal data appears.
Get started
One setup file, then Jimmy sets itself up
Download the latest Jimmy-Setup (about 170 MB) from the project's Releases page on GitHub. It installs for you alone, on any drive, with no Python, Node or admin rights needed. On first start Jimmy checks your PC, asks where to keep your data and how much of the PC it may use, then fetches its own Python and speech model. From 1.0.1 on, new versions install themselves.
- Windows 10 (22H2) or 11, 64-bit
- Lite ~2.5 GB
- Balanced ~4.5 GB
- Full ~11 GB, with local AI
The setup isn't code-signed yet, so Windows shows "Windows protected your PC": choose More info, then Run anyway. NVIDIA GPUs get fast GPU speech; other PCs use the processor.
Questions
Before you install it
Does Jimmy send my screen to the cloud?
Not continuously. Captures are stored on your PC. When you ask a question, the evidence shown with the answer (at most 6,000 characters) goes to the model provider you chose. A question about your screen also sends that window's latest picture, with faces already blurred. Without a key, nothing is sent. The full list.
Does it work without an API key?
Yes. Capture, search, the timeline, insights and cards all run locally. Without a key, each answer shows what search found (exactly what would have been sent to the model) instead of a spoken reply.
Which model providers does it support?
NVIDIA (the default; a free key from build.nvidia.com), Google Gemini on its free tier with NVIDIA as a backup, or OpenAI (paid). Card decisions and meaning search always run locally through Ollama.
Does it record other people?
The mic hears the room, so speech near it is transcribed. System audio, such as the other side of a call, is never captured. Faces in screenshots are blurred, only counts are kept, and the webcam recognises only you, and only after you say "remember my face".
Can it click and type on its own?
Only when you asked, it is sure which control you meant, and the action can be undone. It shows its cursor first and gives you a moment to say "stop". Anything that can't be undone, or a site you didn't name, waits for your "yes". It never types into a password box or acts in banking or password-manager windows.
Will it run on my PC?
Windows 10 (22H2) or 11, 64-bit. An NVIDIA GPU makes speech recognition fast, but it isn't required: other PCs use the processor. Jimmy was built and measured on a laptop with a 6 GB RTX 4050.
Which languages does it understand?
English and Hindi. Hindi works through one model call that also returns the English; its real-world accuracy has not been measured yet.
Does it update itself?
Yes, from version 1.0.1. Ten minutes after it starts and every six hours after that, the installed app checks the project's latest release, downloads a newer setup to your data drive, checks it against the published SHA-256, then asks you to update now or installs it at the next start. One switch in Settings turns this off.
Is it open source?
Yes, MIT licensed. The code, every design decision and the evidence behind it are public (View on GitHub, at the top of this page).