Go one level deeper than 'it's a chatbot'. This topic deconstructs a large language model piece by piece: what parameters physically are, how tokens and training data shape what a model knows, what transformers contribute, how the context window governs what a model can see, what happens during inference, and why hallucinations are a design property rather than a bug you can patch away.
A large language model is a transformer neural network pre-trained on enormous text corpora to do one thing extremely well: predict the next token. 'Large' refers to both the training data (trillions of tokens crawled from the web, code repositories and licensed sources) and the parameter count — GPT-3 in 2020 had 175 billion parameters, and modern frontier models are larger still. A parameter is one learned number inside the network; training adjusted all of them until the model's predictions matched human-written text. After pre-training, models are fine-tuned on instruction-following examples and aligned with human feedback (RLHF) so they behave as assistants instead of autocomplete engines.
Every practical limitation you will hit as an AI user traces back to one of these internals. Token limits explain cost and truncation; training data explains the knowledge cutoff and bias; the transformer explains speed and context handling; inference settings explain output variability; and the prediction objective explains hallucination. When you can name which internal is biting you, you can pick the right fix — split the document (context), ground the model in your docs (RAG), lower the temperature (consistency), or fine-tune the model (task style). Without the internals, every problem looks like 'the AI is broken'.
Follow one request end to end. Your text is tokenised into subword tokens. Embeddings turn tokens into vectors; positional information is added so order matters. The transformer's self-attention layers let every token attend to every other token in the context, and feed-forward layers transform that mixture — parameter weights decide how. At the output, the model produces a probability for every possible next token; a decoding strategy (greedy or sampled with a temperature) picks one, it is appended, and the loop repeats until a stop condition. Everything the model 'knows' beyond this conversation came from frozen training weights; everything it can see right now is bounded by the context window.
Training data (web, code, docs)
Pre-training: predict next Token
LLM = billions of Parameters
Inference: Prompt in Tokens out
Context window = working memory
A concrete example from daily support work: you ask an LLM to summarise a 30-page Intune design document. The doc is roughly 18,000 tokens, so it fits comfortably in a 128k context window (GPT-4o) and the summary comes back accurate. Your colleague tries the same with a 300-page licensing annex — about 190,000 tokens — and the platform either truncates it or errors out. Same model, same question; the context window decided the outcome. Then you ask the model for 'our company's license count' and it invents a plausible number — training data never contained it, and no context supplied it. Each internal component explains one observed behaviour.
Ask an LLM: "What does one parameter in a neural network physically represent, and roughly how many parameters did GPT-3 have vs GPT-4 as publicly known?" Write a one-sentence analogy in your own words — mine: parameters are the memory of every pattern the model ever saw.
Take a real message from your environment, e.g. "Error 0x87D1041C: Application not detected after installation." Paste it into the OpenAI tokenizer and count tokens. Do the math: at roughly 750 words per 1,000 tokens, how many log lines fit in a 128k window?
Paste a long internal document and ask: "Summarise the section about detection rules." Then ask: "Roughly how many tokens have we used so far, and how much of the 128k window remains?" Most models will estimate — the point is to make you think in budgets, not vibes.
Ask the identical question three times: "What does exit code 0x87D1FDE8 mean in Intune?" Compare answers. Sampling with temperature means outputs differ run to run; when you need determinism (scripts, JSON), ask for structured output and state that identical results matter.
Ask for something outside training data: "What is the product code of our internal AppX package?" or "Give me CVE-2033-12345 details." Observe the confident fabrication. Then apply the standard mitigations: "If you do not know, say so and list what evidence you would need." This reframes how you prompt forever.
Three internals carry security weight. First, training data: models can memorise and regurgitate fragments of public data, which is why vendor contracts and no-training guarantees matter. Second, the context window is a trust boundary — anything you paste (logs with device names, user data, secrets) is data the provider processes; treat it accordingly. Third, inference is where prompt injection lives: if your LLM reads untrusted text (emails, logs, web pages), that text can contain instructions. Production designs separate trusted instructions from untrusted content — a principle introduced here and expanded in Level 6.
Parameters are the learned weights — numeric values inside the network's layers that determine how input signals are transformed at every step. During training they are adjusted by gradient descent until the model predicts text well; at inference they are frozen. GPT-3 had 175 billion of them. They encode the model's entire knowledge, which is why a model cannot learn anything new without training or supplying facts in context.
It is the maximum amount of text — in tokens — the model can consider at once: your prompt, any pasted documents, and its own replies share this budget. GPT-4o offers 128k tokens, Claude Sonnet 200k, Gemini up to 1M. It matters because everything the model must reason about has to fit inside it; beyond that you need retrieval strategies, and cost scales with tokens used.
Because the training objective rewards plausible continuations, not verified facts — when knowledge is missing, the model still produces fluent text, which becomes fabrication. Reduction is layered: ground the model with retrieval (RAG) so facts come from your documents, ask for citations and uncertainty, use lower temperature for factual work, and keep a human verification step for anything operationally consequential.
Build a one-page 'LLM internals' card: draw the pipeline (tokens → embeddings → attention → next-token probabilities → decoding), and beside each stage note one IT-work consequence (context limits, cost, hallucination, variability). Then run the two experiments from this topic — tokenizer math on a real log line, and a deliberate hallucination — and write the observed result next to the relevant stage.
The cutoff is the date after which the model has seen no training data — anything newer is unknown, so the model either says so or, worse, improvises. It is fixed by grounding: attaching retrieval (RAG) or web search so current facts are supplied in context at inference time. Vendors also refresh models with newer training runs, but no model is ever current.
Not automatically. Larger windows cost more per request and models can lose accuracy in the middle of very long contexts ('lost in the middle'). Best practice for IT work: keep prompts focused, paste only the relevant log sections, and summarise rather than dump. A well-curated 5k-token prompt often beats a lazy 100k-token dump.
A foundation model is any large model pre-trained on broad data and adaptable to many tasks — it can be text, image, audio or multimodal. An LLM is the text-specific case. GPT-4o, for instance, is multimodal (text, images, audio) — technically a foundation model that includes language capability; a pure text model like an older GPT-3.5 is squarely an LLM.