Agent Workflows in Banking: From Demo to Operations

When we began our first experiments with generative AI agents for Türkiye İş Bankası in 2023, our main question was quite simple: "How can we get AI to do this work?"

As we developed new agents for different business areas, our questions began to change. Was it enough for an agent to produce the right answer? What would happen when the same system ran hundreds or thousands of times in production? How would we detect incorrect decisions? How would we know whether changes to prompts, models, or tools had broken existing workflows? When an agent failed, how would we determine whether the problem came from the model, the data, the tool, or the orchestration?

Over time, we saw that the real problem was keeping agents running safely, observably, and sustainably, rather than simply developing them.

In this talk, I will describe the real journey of the agentic AI systems we developed in a banking environment from 2023 through late 2026. I will share the approaches we tried as we moved from initial prototypes to production workflows, the problems we had not anticipated, the designs that failed, and how these experiences led us to build a shared agent platform.

Throughout the talk, I will explore the following transformation through real examples:

Moving from developing individual agents to shared lifecycle management

The difference between saying "it works" and being truly reliable

How the need for evaluation, observability, and AgentOps emerged

Managing tool, model, prompt, and agent versions together

Why human approval, authorization, and auditability are part of the architecture in banking

How the duplication that arose as each team developed its own agents created the need for a platform

Why debugging deep agents and multi-step workflows becomes increasingly difficult

This is not a story about "how we built the perfect architecture." I will also openly share which assumptions proved wrong, which solutions we had to redesign, and which problems we are still trying to solve today.