Building software platforms has long centered around the human user – their preferences, workflows, and comfort with existing tools and languages. However, the rise of artificial intelligence as the primary user of these platforms is fundamentally altering this perspective. It’s prompting a re-evaluation of product requirements and, consequently, changing how we build software and for whom we build it. The domain of software testing platforms offers a compelling, concrete case study of this dynamic shift.
Traditional Software Testing: Built for Humans
Consider traditional software testing tools, many of which were designed with human engineers in mind. Platforms like Selenium or Appium support a multitude of programming languages, allowing testers to work within familiar environments. They often include features like recording tools to capture human interactions and debuggers to help step through tests and inspect application structure (like the DOM). The typical human workflow involves creating a test, manually refining it, fixing issues like suboptimal selectors, and investing significant effort to make tests robust against application changes because creating and maintaining them is time-consuming and costly.
“For an AI user, they are comfortable with everything.”
AI as a User: A Different Set of Requirements
When the user is an AI, many of these human-centric features become less relevant or even counterproductive. An AI doesn’t have a preferred programming language; it can adapt to various formats. The fundamental goal of the development language or environment shifts from prioritizing expressiveness, ergonomics, and human generation to creating something distilled and robust that can be fairly easily understood by humans (for auditing) and is easy to execute, even if it wasn’t generated by a human. This change in focus indirectly leads to approaches like declarative languages or domain-specific languages (DSLs), which facilitate the reuse of test components through declarative libraries and help the AI efficiently adapt tests when the application’s underlying structure changes.
“Having the AI drive the recording tool would be an unnecessary error prone step. So test generation tools become totally obsolete.”
Furthermore, an AI doesn’t require recording tools in the human sense. Instead, it can directly interact with an application, capture visual information (screenshots), analyze the structural data (the DOM), and iteratively construct a test by understanding both the visual and underlying elements. This process can drastically accelerate test creation, potentially reducing the time from 20-30 minutes for a human to as little as 30 seconds for an AI.
The reduced cost and increased speed of AI-driven test generation also lessen the pressure for individual tests to be hyper-robust against minor application updates. While some level of robustness is still beneficial, the ability to quickly regenerate or update tests makes managing changes, even updating a dozen tests weekly, a much more manageable task.
| Consideration | Human User | AI User |
|---|---|---|
| Preferred Language/Environment | Highly important (familiarity) | Not important (adaptable) |
| Recording/Generation Tools | Important (initial test creation) | Obsolete (direct interaction) |
| Debugging/Analysis Tools | Important (manual refinement, DOM inspection) | Less important (processes raw logs) |
| Test Robustness (against minor changes) | Highly important (tests are costly to create/maintain) | Less important (tests are cheap to regenerate) |
| Abstraction and Reuse | Beneficial | Benefits from well-designed abstraction (e.g., declarative libraries) |
| Reporting and High-Level Dashboards | Necessary (oversight) | Necessary (analysis) |
| Simple, Efficient Authoring (Guardrails) | Beneficial (understandability) | Beneficial (prevents complex, error-prone outcomes) |
| Final Artifact Readability | Important (auditing, understanding) | Important (human auditing) |
Reporting and Simplicity: Common Ground
Reporting and high-level dashboards remain important for both human oversight and AI analysis to gauge the overall health of the software. However, the AI doesn’t necessarily need the same sophisticated graphical interfaces for detailed analysis and diagnostics; it can efficiently process raw logs to understand why tests failed.
Interestingly, while AI can handle complexity, the testing example highlights that even AI users benefit from well-designed, simpler tools. Providing an AI with too much flexibility, such as the ability to execute arbitrary JavaScript within a test, can lead to overly complex and error-prone outcomes. This suggests a common ground: for both human and AI users, designing platforms that guide towards simple, efficient, and understandable test authoring is advantageous. The final test artifact, even if AI-generated, should ideally be readable for human auditing and comprehension.
Implications for the Future of Software Building
This case study in software testing illustrates a broader point: traditional, off-the-shelf software solutions built primarily for human workflows may face disruption. New tools optimized for AI interaction can be significantly more effective and efficient. Moreover, the testing example suggests these AI-centric platforms can be built relatively quickly and cost-effectively by companies for their specific needs (potentially in under a week by one engineer), shifting the “make versus buy” calculus towards building custom solutions.
“The tools that result are 10 to 100 times more effective and efficient.”
Ultimately, AI is changing the client of our software platforms. We are moving from building primarily for human engineers to building for AI or “cyborg” users. This doesn’t necessarily mean fewer jobs, but rather a transformation in focus. Instead of engineers spending time navigating complex third-party tools, they can dedicate their efforts to building powerful, custom platforms that leverage AI to achieve superior and more tailored results. Software development is evolving to be about creating the tools that empower AI to build and test software more effectively.
lloyd