Your family agent needs a butler's book
I've always struggled with "life admin" work. And becoming a parent made it harder: more random stuff, less free time.
In theory, the answer is: spin up a family AI agent that takes care of the busywork. But in practice, it's easy to spend more time building the system than actually getting value from it!
Over the past few months, though, I feel like I've finally landed on a family agent that really works. It handles weekly meal planning and orders groceries; it can help with routine emails and scheduling; it can answer quick questions like “Did we schedule that appointment yet?”; and it is doing a lot of the work for planning an upcoming move.
What's the secret? Well, as with so many things in life, it's about nailing the basics: the agent needs the right context. I call this context a "butler's book", as an analogy to the old way of assembling context about a household:
Back in Victorian times, the Butler’s Book was where the butler kept all the information about the household: staff and maintenance schedules, inventory of china and silverware, food preferences, measurements for clothing, etc.
In this post I will show you how I've assembled a modern-day butler's book, and I hope you take away some ideas for constructing your own.
(This post is Certified Organic. No AI was used to write it; 100% typed with my own two hands.)
What's in the book
A Butler's Book is a curated set of knowledge about your family, including background, routines, and projects. Some examples of documents in ours:
When your agent has access to all of this information, it feels like it actually "gets you" and can quickly provide value without you needing to explain everything from scratch every time.
For example, I can ask "any gear maintenance tasks I need to do?" and my agent can search all my appliance manuals and tell me "clean the air filters and adjust the car seat". I can also set up proactive reminders for this on a timer.
Or I can say "plan meals for this week" and my agent can suggest some recipes we like, without repeating last week's plan. The results feel hyper personalized and not generic. We're never starting over from scratch.
Designing your book
IMO there are a few considerations when designing a Butler's Book and picking your tools.
In terms of specific tools: my family uses Notion as our butler's book. We store docs in a shared workspace, and we use the built-in Notion AI agent. This has actually replaced most of my usage of the ChatGPT/Claude apps, since I prefer talking to an agent that lives closer to my context. (Disclaimer: I'm very biased on this point since I work at Notion!)
There are many other options for tools, such as OpenClaw. Whatever tools you use, the core idea of assembling a butler's book remains the same. One thing I would strongly encourage for family use cases is to think about collaboration up front -- some tools assume a single user and don't scale well to even two people that want to share context.
How to use the book
Here a few examples of fun and useful things our butler's book has enabled.
We're moving this summer. I'm auto-importing Zillow search results into a database daily, so that my agent can see all the listings too. The sync was built for me by a coding agent and runs in the cloud.
Of course my agent also knows the broader context of our move and exactly what criteria we're looking for. So now I can just ask it "any promising listings recently?" and it can give me a shortlist that actually warrants further review. So much easier than scanning all of Zillow manually. (I also have it autofilling notes on every new entry so I don't even need to ask.)
Another example: recently my agent drafted a spreadsheet budget, a powerpoint presentation overview for me to share with my wife, and a written doc outlining the plan. Crucially all of these artifacts are now context for future conversations -- I can ask my agent the sorts of boring tax questions I would ask a financial advisor, and it already has all the necessary background.
Finally, weekly meal planning has been a great time saver. Every weekend I'll spend a couple minutes asking my agent to draft a meal plan for the week. It follows a specific process: checking our recipe library, looking at recent meals to avoid duplicates, thinking ahead about the weather this week. Of course my wife and I also contribute ideas to the plan, but having the agent suggest a starting point is really helpful when we're busy with kid stuff.
Not only that, my agent then also proceeds to buy all the ingredients on Instacart for me! This uses a custom agent tool which makes some calls to @browserbase to click around in a browser. (Notion also recently launched the ability to deploy arbitrary custom tools to use with your agent.)
Context isn't just for companies
The idea that context matters for AI is very mainstream at this point. But I think most of the discussion revolves around either bigger companies or single individuals.
Meanwhile, a family / household is a really valuable place to assemble context for an AI. There is a ton to keep track of, and time is precious. Spending less time in email and more time with my family is one of the greatest rewards imaginable.