LLM Tester: quality assessment of AI systems

Testing and quality assessment of AI systems is one of the fastest-growing niches in IT, and one with a low entry barrier. On this hands-on course you learn to evaluate systems built on large language models (LLMs) — from manual, rubric-based response evaluation to automated eval pipelines, RAG and AI-agent testing, and vulnerability discovery (red teaming).

No prior programming background is required: the course opens with a Python foundations block (data types, loops, functions, files, packages), then moves to applied work — calling LLMs via API (OpenAI, Anthropic), JSON test datasets and pytest.

Core areas, with concrete tools:
• how LLMs work — tokens, context, non-determinism; defect classes (hallucinations, bias, prompt injection);
• prompt design and evaluation rubrics; manual assessment — side-by-side comparison, ranking, failure-mode reports;
• quality metrics and the LLM-as-a-judge method;
• automated eval pipelines (promptfoo, DeepEval, pytest), golden datasets, CI and regression runs;
• testing RAG and agents — the Ragas approach, tool-use correctness, tracing (LangSmith);
• basic red teaming — prompt injection, jailbreak tests, guardrails.

The course is built around practice and ends with an end-to-end project on a real LLM application (rubric + automated eval run + red-team pass + quality report) — a portfolio piece and genuine readiness to start work. AI evaluation platforms hire internationally, and English-speaking QA roles exist in local product companies.

An advantage of the course is its full classroom (on-site) format in Tallinn; real-time online participation is available if needed.

Gamma Intelligence OÜ passed the HAKA quality assessment of the «tarkvara ja rakenduste arendus ning analüüs» (software and applications development and analysis) õppekavarühm in continuing education (täienduskoolitus) in 2023. More: https://haka.ee/taienduskoolitus/oppekavaruhma-kvaliteedi-hindamine/