I run a lot of A/B tests. And for a while, my documentation for past tests was scattered across notes, screenshots, and random repo files. I wanted something cleaner, so I built a showcase app where I could document tests, toggle between control and variant experiences, and see how each change played out.
The idea seemed pretty simple. It got complicated fast, which honestly made it more interesting.
What I wanted to build
The goal was a system that could support any A/B test without touching the core app structure every time I added a new one. Test details (the control, the variant, the methodology, the results) would live in a central data file. The test detail page would pull from that file and render itself accordingly.
No hardcoded per-test pages. Just one flexible template that adjusted based on whatever test you were looking at.
First hurdle: state management
My first version had a wrapper component handling the toggle between control and variant. That caused the toggle button to render twice, once in the wrapper and once in the test page itself, which was already managing its own state.
Once I realized the test page didn't need the wrapper at all, removing it cleaned everything up immediately. Sometimes the fix really is just deleting code.
Second hurdle: injecting build code
Each test has a function that manipulates the page when the variant gets triggered. The tricky part: Next.js doesn't love dynamic DOM manipulation, and it was easy to end up in a situation where the build code fired too early, or didn't fire at all when the toggle was clicked.
The fix was making sure the variant code only ran when explicitly triggered, and stepping outside the React rendering cycle to do it. It's not the most elegant solution, but it worked reliably once I got the timing right.
Third hurdle: ESLint and Vercel
The build functions are self-invoking scripts that modify the DOM directly, which TypeScript really doesn't like. It flagged pretty much everything with implicit type errors. Vercel wasn't happy either.
Honestly, I just silenced the linter for that file. Getting it working was more important than getting it perfect, and I can revisit it later if it actually becomes a problem. So far it hasn't.
What I took away from it
A few things stuck with me after this one.
Keeping test data in a separate file is genuinely worth the upfront effort. The test detail page is agnostic now. Adding a new test is just adding an object to the data file.
State management gets messy fast when multiple layers are trying to manage the same thing. The wrapper component wasn't doing anything the test page wasn't already doing, and it was actively causing problems. Removing it was the right call.
And Next.js has opinions about the DOM. If you need to manipulate it directly (which A/B test code injection kind of requires), you have to be intentional about when and how you do it.
What's next
I want to integrate Google Tag Manager so I can track actual results inside the app, and build a dashboard to visualize performance across tests. Right now it's mainly a documentation and demo tool, which is useful, but the data layer would make it a lot more interesting.
If you're thinking about building something similar, hopefully this saves you a few headaches on the weird parts.