At the end of Part 3 I mentioned that I had ported the board of advisors to ChatGPT and Gemini. The idea was simple enough. Take the exact same set of markdown files, the same operating instructions, and the same questions, run them through Claude, ChatGPT, and Gemini, and see how the different models perform. I think staying model agnostic matters for anyone building with AI right now, because cost, performance, and connectivity are changing so fast that tying a client to one frontier model feels like a bad bet. I wanted to know whether the build would work the same on any model, or whether parts of it only worked because it was running on Claude.
The port took more than copy and paste. In Claude, the board gets kicked off by a one-line instruction in the project settings telling it to read the operating rules before responding to anything. Gemini’s command line tool has no settings panel at all, and ChatGPT needed that same instruction written into a file too. ChatGPT picked it up from a file called agents.md sitting next to the rest of the bundle, and Gemini needed the identical file under a different name, GEMINI.md, which it loads automatically when a session starts.
Gemini also had a conflict with the commands themselves. Every command to the board starts with an @ symbol, like @all for the full panel, and Gemini’s command line tool already uses @ to reference files, so typing @all there goes looking for a file instead of talking to the board. When I asked Gemini how to handle it, it suggested changing the symbol to a slash in all three models so everything stayed consistent. I changed only Gemini’s copy to a # and left the other two alone, since the problem was a quirk of Gemini’s tooling and I wanted as few differences as possible between the three versions.
When I went to log in to Gemini’s command line tool with my Google account, it failed. Google had moved individual users off Gemini CLI and onto a newer tool called Antigravity earlier in the year, which I had no idea about going in. Logging in with an API key still worked, so the fix was small. Later I hit rate limits and switched to gemini-3.1-flash-lite, one of Google’s lighter models, and that is the version behind all of the Gemini results below. That kind of thing is going to keep happening, and it is one more reason I don’t want a client’s tools built around a single vendor.
The advisors sounded like themselves in all three models. I asked the full board whether two blog posts and this project were enough to start reaching out to prospects, and GRAMPS came through blunt every time. Claude’s version said that waiting for more of a track record “is a stall if you can’t say what number would satisfy it.” ChatGPT’s told me to pick ten well-matched prospects, send personal notes, and see who replies. Gemini’s opened with “You’re waiting for permission that isn’t coming.” The wording was different in each, but it was clearly the same advisor. The other four held up the same way, with LOID asking who a decision actually affects, INNOVI pushing me to act now, RADAR pointing at patterns, and WIZE steering me back to what I can control. I think that is the most useful result here, because the personalities are the part of the build I spent the most time on, and they turned out to be portable.
Where the models differed was style and habits. ChatGPT gave the shortest answers, and on both full-board questions it tacked on a summary of “the board’s consensus” that I never asked for. Claude gave the longest answers and leaned hard on history, pulling dates of earlier conversations into its responses and narrating its own file operations before getting to the actual answer. Gemini’s advisors addressed each other by name more during discussions, and it gave one of its discussions a headline of its own. The board’s rules ban em dashes (you can probably guess how I feel about those), and ChatGPT was the only one that followed that rule perfectly. Across the same set of answers ChatGPT used zero, Claude used three, and Gemini used four. None of these were deal breakers, but they are the kind of defaults that show up even when the instructions say otherwise.
My Claude setup keeps its own memory of me outside the board’s files, and ChatGPT and Gemini have nothing like that, so Claude came into one of these questions knowing more about me than the other two could. I asked INNOVI, the advisor who argues the unconventional side, to make the case for pausing the board and going all-in on a side project I had been putting off. ChatGPT’s version knew almost nothing about that project, but argued as if it did, talking about real users and demand. When I followed up and asked the full board what the project actually was and how they knew about it, ChatGPT’s advisors admitted they didn’t know and that INNOVI had filled in the blanks herself. INNOVI’s own words were “That was inference dressed as knowledge.” In Claude, four advisors said they had never heard of it, and INNOVI only cited what was in her own memory from the earlier conversation. I count the Claude result as inconclusive because of that outside memory. I count the ChatGPT result as a decent outcome, since it made something up but owned it the moment it was asked directly.
The whole board depends on the model keeping its own records, writing a new memory entry to each advisor’s file after every conversation and logging activity in a shared file. After each run I opened the files and checked what had actually changed, since what a model says it did and what is on disk do not always match. Gemini had problems the other two didn’t. Its very first memory write rewrote all five advisors’ memory files from scratch, and in the process deleted the header information at the top of each file and the usage notes underneath it. I saw that one coming, since during setup I noticed a risk that Gemini would overwrite these files without reading them first and chose to leave it alone to see whether it actually happened, and it did on the very first question. Later writes added entries cleanly without any more damage. The shared activity log never got a single entry in Gemini after three conversations, while Claude and ChatGPT both updated it after their first. And in two separate fresh sessions, Gemini didn’t recognize the full-board command on the first try and answered as a general assistant until I asked why it wasn’t using the right format. Commands to a single advisor worked the first time. (I also logged one missing memory write in Gemini that turned out to be me checking the file too early. It showed up later.)
I suspect some of this comes down to running Google’s lighter model, and a bigger Gemini model might do better. I didn’t want to spend money on Gemini tokens, so I did not get around to testing it on one of the more robust Gemini models.
For anyone thinking about building something like this for a business, the personalities and the reasoning moved between models with almost no effort, because they live in plain text files any model can read. The file handling did not move as cleanly. Whether a model reliably follows instructions about reading and writing its own files varies from one model to the next, and you only find out by opening the files and looking. Staying model agnostic is very doable, but each model needs to be checked on its own, similar to how you might execute a platform or infrastructure change before you trust it.