Skip to content
Pencil

Agent Architect

EMEARemote · EMEA
Staff · FullTime · Product
POSTED 17 JULVERIFIED 14H AGO
Apply Ask
Role
114 days opentrue age
Posted 17 Jul
CompanyPencil →
28open roles
Opened · 30d
+0
Closed · 30d
−0
30d window
Market
23,354open · Staff Other
All other
6,25,210open
All tracked boards · live 8 Nov · n=23,354
LLMs

About us

Pencil uses generative AI to make advertising faster, cheaper and better performing, without replacing the creative people who make it. We are building the agentic operating system for marketing: Scribble, our agent, takes a brief and produces finished, brand-safe creative across image, video and structured formats for global brands and small businesses alike.

The role

Agent Architects take a client's creative problem and make Scribble solve it reliably. You will lead discovery with brand and agency teams, define with them what a correct output looks like, and improve Scribble's performance against that definition until the work is ready to ship. You will then work out which parts of that solution should become standard behaviour in the core product for every customer.

This is a consulting role. You will shape agent behaviour through Pencil skills, evaluation criteria and a structured optimisation process, and you will build small helper projects, such as scripts, prototypes or data preparation tools, where they speed up the work. Where a client problem needs a change to the platform itself, you will make the case to the core product squads with evidence from your engagements.

You will report to the Head of Agent Architecture and be deployed into client teams as engagements require.

How we work

Our optimisation practice rests on a few principles. We start from the client's real briefs and agree what good looks like in terms precise enough to measure. We measure performance before changing anything, so every improvement can be shown against a baseline. We change one thing at a time and keep a record of what changed and why. We treat a failure as something to diagnose before fixing, because the cause may sit in the inputs, the agent's instructions or the way we are measuring. We call the work done only when results hold up on repeat. Much of the mechanical work is carried out by AI agents that you direct, so your time goes on judgement, diagnosis and the client relationship.

Key responsibilities

Client discovery: Run structured discovery sessions with brand and agency stakeholders, and turn ambiguous briefs into a scoped problem with an agreed, measurable definition of done.

Optimising for clients: Lead the optimisation of Scribble for your engagements from baseline to confirmed result, and give clients clear evidence of what improved and why.

Measurement discipline: Design evaluation criteria, check that they catch known failures before relying on them, and keep them honest as the work reveals cases they missed.

Extrapolating to the core: Generalise what works for one client so that it serves a whole category of work. Contribute examples, evaluation criteria and findings to the relevant core squad, and write clear, evidenced requests where a client problem can only be solved in the platform.

Reusable practice: Build and maintain a library of skills, evaluation patterns and engagement playbooks that make each new engagement faster than the last.

Communicating trade-offs: Explain results, failure modes and trade-offs between cost, speed and reliability to non-technical stakeholders in plain language.

Your background

At least three years of hands-on work with large language models and generative image or video models, with a practical understanding of how agents fail and why.

A background in solutions architecture, technical consulting or forward-deployed engineering, or a background in creative or design work where you have applied structured, critical thinking to technical problems. What matters is that you can take a vague problem, break it down and test your way to an answer.

Enough technical understanding to work confidently alongside engineers. You can read JSON and API documentation, follow how data moves through a system, and direct AI agents to carry out multi-step work. Writing a simple script is a plus.

An evaluation habit. You do not trust an output you have not measured, and you are comfortable designing scoring criteria and defending them.

Source · PencilFirst seen 7 NovOriginal ↗