Research proposal · Intelligent Robot Learning Lab · University of Alberta
My vending robots, a vision-based store and a team of LLM agents run seven small businesses. Each needs my approval, and I am rarely available. How can these agents learn my reward from ratings and corrections, measure how aligned they are with me, and decide what to do while I am not watching?
Why this lab, why me
For 20 years I have built businesses that must keep running when I am not there: a distribution company, rental properties, vending machines, and now AI agents. Every one of them hit the same limit: decisions waited for me. That is the problem this lab studies from the other side, how an agent learns from a human who teaches it, and how much it can do alone.
Data migration engineer. Designed SAP data-migration processes (legacy extraction, cleansing and transformation rules, SQL) for manufacturing plants.
Founder, LED lighting distributor. Grew it to about 200 employees and US$6–7M in yearly sales, with offices across Mexico and in China. Learned how hard it is to delegate decisions.
Real estate investor. Built and ran rental and short-term rental properties, managed at a distance.
Canada. Vending robots, AI agents for content, SEO, investing and sourcing, a cashierless vision store, and Applied AI & Machine Learning at Conestoga College.
A deployed testbed
Most human-in-the-loop learning is tested in simulation or in short lab studies. This testbed runs every day, earns real money, and is taught by one real owner whose availability is limited and measurable.
Deployed at a clinic, an auto-parts yard, a hotel and a car dealership, with sales telemetry. The owner corrects every restock plan; his rules cover his time and driving distance, which the textbook optimum ignores.
GrabGo turns a shelf into a store that charges from images: a shelf camera, a customer's phone photo, or both. The owner's privacy and cost trade-offs decide the setup at each location.
Language-model agents propose posts, site changes, market theses and supplier decisions. Every approval, rejection and correction with its reason is logged.
site changes from the SEO agent ended their first week waiting for the owner. Waiting, not error, was the main cost.
Other people sometimes approve on the owner's behalf. Their answers are a second, misaligned teacher: in simulation, learning from them replaces the owner's profile without anyone noticing.
The owner is in class most weekdays. His availability is a real, scheduled monitor: feedback exists only when he is watching.
Where this meets the IRL Lab
I read the lab's recent work and the research interests listed on its team page. These are the points where my data and my operation could be useful, member by member.
Experiment 1 · building on R4 and TAC
The owner reviews spending proposals. With a preference, he picks the better of two proposals. With a rating, he labels one proposal bad, neutral or good. Both cost him one answer. I fit a linear reward from each kind of feedback (a ranking-MSE loss in the spirit of R4 for ratings, Bradley–Terry for preferences) and score it two ways: alignment with the owner (Kendall τ-b, as in TAC) and the share of proposals the agent would approve or reject differently from him.
Alignment = Kendall τ-b between the owner's true utility and the learned reward on 600 new proposals (1 = same ranking). 20 runs per point.
Share of new proposals where the agent's approve/reject differs from the owner's. This owner approves about a third of proposals.
Both kinds of feedback learn the owner's ranking well; preferences catch up with ratings after about 160 answers. But preferences never tell the agent where the owner's bar is: even with 160 answers the agent still approves or rejects about 19% of proposals differently from him, against about 5% with ratings. For an agent that must decide whether to act, ordinal ratings carry the information that matters. This supports R4's motivation in a non-robotic, money-valued domain.
Experiment 2 · the owner as a monitor
Decisions arrive every hour, because vending robots and an online store never close. The owner is available a few hours a week. Only then does he review the queue (up to six items an hour) and rate them; the agent refits its reward every week from the ratings it has seen. Three policies: always wait for him, always act alone, or act alone only when the learned reward is confident and the action is reversible.
Average hours a decision waits for the owner (log scale), 12 weeks, 8 runs per point. Below about 30 hours a week the queue never clears when the agent always waits.
When the owner is the monitor, waiting is the dominant cost, not error. Acting alone on everything makes about 7 to 13 mistakes per 100 decisions. Acting alone only on confident, reversible decisions (about a quarter of them) keeps mistakes near 0.3 per 100 and, at 20 owner-hours a week, cuts the average wait from about 290 to 65 hours. Choosing which decisions to keep for the owner, and when to ask him, is exactly the open question of learning under a monitor.
Proposed MSc thesis
A live system the lab could study without building it: robots in the field, a vision store, LLM agents, logs and a real teacher.
25 years founding and operating companies (up to about 200 employees, offices in Mexico and China), plus SAP data-migration engineering. I can run an applied project end to end.
I am open to a funded position or an unfunded research collaboration.
Both experiments use a simulated owner built from the portfolio's decision profile (case 01); the comparisons, not the absolute numbers, are the result. Code: github.com/antoniosg2025-stack/owner-in-the-loop
Projects I am working on
Each case is a real operation with its own notebook and one main finding. Together they are the evidence behind this proposal. The full interactive portfolio is at research.antonio-sainz.com, and the code is on GitHub.
One decision profile per type of decision makes a third of the errors of a single profile and asks half as often.
Open the case → 02 · Equilibra$8,800misplaced per user when copying the owner's money moves; learning his rule with 13 questions fixes it.
Open the case → 03 · Press4× fasterlearning from a correction with a reason than from yes/no approvals.
Open the case → 04 · SEO360 → 126questions to the owner in 12 weeks when only reversible changes run alone.
Open the case → 05 · Vending$83 → $155weekly margin after travel, because the owner's rules count his time and driving.
Open the case → 06 · GrabGo−90%money lost to image-recognition errors when look-alike products share a price.
Open the case → 07 · Suppliers15% → 2.5%of lots where agent and owner disagree, once his tolerance is learned per product.
Open the case →References