Skip to content

About

Eight hands-on demos of NVIDIA Nemotron 3 Nano running locally with Ollama: reasoning on/off, tool calling, structured output, long context, notes Q&A, code review and a chat API.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

1 Commit

Folders and files

Repository files navigation

Nemotron local lab

CI Python Model Runs on License: MIT

Eight small, hands-on experiments with NVIDIA Nemotron 3 Nano running on your own machine through Ollama. Each demo is one short script, prints something you can judge for yourself, and is built on a tested module you can reuse.

Nothing leaves your machine: no API key, no cloud, no cost per token.

The demos

# Demo What you see
1 01_hello.py The reasoning trace and the answer streaming separately, then prefill and generation speed
2 02_think_toggle.py Eight puzzles with reasoning off and on: accuracy, tokens and time side by side
3 03_tool_agent.py An agent that lists and reads invoice files and adds them up with a calculator tool
4 04_structured.py Messy support emails turned into validated Pydantic objects, scored field by field
5 05_long_context.py A needle-in-a-haystack test at several prompt sizes and depths
6 06_ask_notes.py Questions answered from a folder of Markdown notes, with citations that are checked
7 07_code_review.py A diff reviewed into structured findings, scored against four planted bugs
8 08_chat.py A small chat API that streams reasoning, answer and speed as JSON lines, for a chat UI to sit on

Quick start

You need Ollama and uv.

ollama pull nemotron-3-nano:30b       # 18 GB; see "Which model" below
git clone https://github.com/gsaini/nemotron-local-lab.git
cd nemotron-local-lab
uv sync
uv run demos/01_hello.py

Every demo takes --help. To try another model, set NEMOTRON_MODEL, for example NEMOTRON_MODEL=nemotron-3-nano:4b uv run demos/02_think_toggle.py.

Which model

Tag Download Good for
nemotron-3-nano:30b 18 GB The default. Mixture-of-experts: about 3B of its 30B parameters run per token, so it is quicker than its size suggests. Wants a machine with 32 GB of memory.
nemotron-3-nano:4b 2.9 GB Laptops with 8–16 GB. Much faster, and weaker on the puzzles and the agent.
nemotron:70b 40 GB or more The older Llama-based Nemotron. Needs 64 GB or more.

What each demo teaches

1 · Hello. Nemotron 3 Nano is one model for both reasoning and quick answers. Ollama returns the reasoning in its own field (message.thinking), so the demo shows it dimmed and the answer in full. The speed line separates prefill (reading your prompt, fast) from generation (writing, slower): that split explains most of what you feel when a local model is "slow".

2 · Reasoning on vs off. The same puzzles with think=False and think=True, graded by script from a final ANSWER: line. Reasoning tends to fix trick questions and costs many more tokens. Run it and decide where it is worth it.

3 · Tool calling. The loop in agent.py is the whole idea of an agent: send the tools, run what the model asks for, send the results back, stop when it answers. The calculator only accepts arithmetic, and the file tools can't leave their folder, so a confused model can't do harm. The answer needs four files and a part payment: 4175.75 is still owed, and Globex owes the most (2075.00).

4 · Structured output. Ollama can constrain decoding to the JSON schema of a Pydantic model (format=Ticket.model_json_schema()). When a model build doesn't support that, extract() puts the schema in the prompt instead and says which mode it used. Either way we validate, and on failure send the error back once. The expected labels are in data/emails/expected.json; severity is a judgment call, so treat those misses as debatable.

5 · Long context. One line hidden in a synthetic log, near the start, middle and end, at three sizes. The demo sets num_ctx for each run: Ollama's default window is small, and a prompt that doesn't fit is cut from the front without an error. That is the most common reason a local model "ignores" your document.

6 · Ask your notes. BM25 keyword search (no embedding model to download), the top chunks numbered as sources, and an answer that must cite them. The demo flags citations that point at no source. Point it at your own notes with --dir; they never leave your machine.

7 · Code review. A diff goes in and a Review object comes out: file, line, severity, issue and fix for each finding. On the sample diff it is scored against four planted bugs. Pipe in your own change with git diff | uv run demos/07_code_review.py -.

Results on my machine

Measured on an Apple M1 Pro with 32 GB, Ollama 0.40.11, nemotron-3-nano:30b (18 GB download, NVFP4, run by Ollama's MLX runner), November 2026. One run each, so treat the numbers as a first look.

Demo Result
1 · Hello Correct three-sentence answer. 43 tokens/s generation. The first call took 365 s because it loaded the 18 GB model into memory; later calls start at once.
2 · Reasoning on vs off Off: 7/8 correct, 353 tokens, 12 s. On: 8/8, 2,502 tokens, 59 s. Reasoning fixed only the letter count, and cost about 7× the tokens and 5× the time.
3 · Tool agent Correct: 4175.75 still owed, Globex owes the most (2075.00). It listed the folder and read all four files, but did the sums in its head instead of calling calculate.
4 · Structured output Not measured yet. Ollama answered structured output is unavailable (HTTP 501) for this model build, so extract() now falls back to putting the schema in the prompt and validating the reply. The fallback is tested offline, not yet against the model.
5 · Long context Not run yet.
6 · Ask your notes Correct (400 euros) and cited the right source, but wrote the citation as 【1】, not [1]. The checker now accepts both.
7 · Code review Not measured yet, for the same reason as demo 4.

Two things these runs taught, both now handled in the code:

  • Constrained JSON output isn't available for every model build. Check before relying on it, and keep validating either way.
  • Models have their own habits. Nemotron's 【1】 citations would have scored as "no citations" with a strict [1] check.

Layout

src/nemolab/
  llm.py        LLM.chat / LLM.stream over Ollama, one Reply type with token counts and speed
  fake.py       a scripted stand-in model, so the tests run without Ollama
  puzzles.py    the eight puzzles and the answer grader
  agent.py      the tool loop, a safe calculator, a read-only folder
  extract.py    schema-constrained extraction with validation and one retry
  haystack.py   synthetic logs with a hidden line
  notes.py      Markdown chunking, BM25, numbered sources, citation checks
  review.py     the Review schema and planted-bug matching
  chat.py       a standard-library HTTP server: GET /api/info, POST /api/chat streaming JSON lines
  show.py       terminal tables and colours
demos/          the eight scripts
data/           invoices, emails with expected labels, sample notes, a diff with planted bugs
tests/          33 tests, all offline

Tests

uv run pytest -q          # 33 tests, no model needed
uv run ruff check . && uv run ruff format --check .

The tests check the plumbing (requests, the agent loop, retries, grading, retrieval, the sample data) with a scripted stand-in model. They say nothing about how good Nemotron is. Only the demos measure that, and only on your machine.

Limits

  • Each demo is a handful of examples, not a benchmark. Results change with the model tag, quantisation and Ollama version, and can vary between runs even at temperature 0.
  • The puzzles are well known, so the model may have seen them in training.
  • Speeds depend heavily on your hardware and on what else is running.

License

MIT

About

Eight hands-on demos of NVIDIA Nemotron 3 Nano running locally with Ollama: reasoning on/off, tool calling, structured output, long context, notes Q&A, code review and a chat API.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages